2024-04-25T00:41:50.184 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T00:41:50.184 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285 --owner scheduled_teuthology@teuthology --description fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/orig.config.yaml 2024-04-25T00:41:50.384 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:41:50.487 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285 branch: squid description: fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672285' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4w3XTBD3z3iI/FJ0TDJz24aQSOg0wgpZEJWbM2zwmoU5cppJBwtxtTf2VrZri4GLiwfDKYSxwe32Eb7UzdZk8= smithi187.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNMI9STWpOkq+7tF/jBvaEKwDNQ9dzEZTLukGTCCNQDFKQkri1l5VVzrOTwYMlImxhfXnlgrD3G/FfhHvX3p0Kg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cephfs_shell teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T00:41:50.488 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T00:41:50.488 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T00:41:50.489 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T00:41:50.490 INFO:teuthology.task.internal:Checking packages... 2024-04-25T00:41:50.515 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T00:41:50.515 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T00:41:50.515 INFO:teuthology.packaging:ref: None 2024-04-25T00:41:50.515 INFO:teuthology.packaging:tag: None 2024-04-25T00:41:50.516 INFO:teuthology.packaging:branch: squid 2024-04-25T00:41:50.516 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:41:50.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-25T00:41:50.727 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:41:50.728 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T00:41:50.735 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T00:41:50.735 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T00:41:50.740 INFO:teuthology.task.internal:Saving configuration 2024-04-25T00:41:50.758 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T00:41:50.764 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:41:50.784 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:36:06.856315', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKh2UGv1J5fO9otLt9nVB3AnVFjiEBMZ96JQgkeujqv6'} 2024-04-25T00:41:50.801 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:36:06.858043', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICpSaT6DblHgEplKMm96x4XZc6KqLNJ7H24eJc1VkIiL'} 2024-04-25T00:41:50.801 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T00:41:50.809 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-04-25T00:41:50.809 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-04-25T00:41:50.809 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T00:41:50.869 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f372dc383a0>, signals=[15]) 2024-04-25T00:41:50.870 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T00:41:50.876 INFO:teuthology.task.internal:Opening connections... 2024-04-25T00:41:50.876 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:41:50.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:41:50.960 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:41:50.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:41:51.042 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T00:41:51.049 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2024-04-25T00:41:51.054 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2024-04-25T00:41:51.054 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2024-04-25T00:41:51.102 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T00:41:51.102 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="Ubuntu" 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="22.04" 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_CODENAME=jammy 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:ID=ubuntu 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE=debian 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T00:41:51.103 INFO:teuthology.orchestra.run.smithi112.stdout:UBUNTU_CODENAME=jammy 2024-04-25T00:41:51.104 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2024-04-25T00:41:51.126 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2024-04-25T00:41:51.131 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2024-04-25T00:41:51.132 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2024-04-25T00:41:51.179 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T00:41:51.179 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="Ubuntu" 2024-04-25T00:41:51.179 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="22.04" 2024-04-25T00:41:51.179 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T00:41:51.180 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_CODENAME=jammy 2024-04-25T00:41:51.180 INFO:teuthology.orchestra.run.smithi187.stdout:ID=ubuntu 2024-04-25T00:41:51.180 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE=debian 2024-04-25T00:41:51.180 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T00:41:51.180 INFO:teuthology.orchestra.run.smithi187.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T00:41:51.180 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T00:41:51.180 INFO:teuthology.orchestra.run.smithi187.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T00:41:51.180 INFO:teuthology.orchestra.run.smithi187.stdout:UBUNTU_CODENAME=jammy 2024-04-25T00:41:51.180 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2024-04-25T00:41:51.201 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T00:41:51.210 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T00:41:51.215 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T00:41:51.215 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:41:51.218 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:41:51.227 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T00:41:51.235 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T00:41:51.235 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:41:51.263 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:41:51.286 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T00:41:51.364 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T00:41:51.386 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T00:41:51.386 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T00:41:51.386 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:41:51.387 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:41:51.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:41:51.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:41:51.392 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-04-25T00:41:51.393 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-04-25T00:41:51.439 INFO:teuthology.orchestra.run.smithi112.stdout:5.15.0-56-generic 2024-04-25T00:41:51.439 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.15.0-56-generic 2024-04-25T00:41:51.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get clean 2024-04-25T00:41:51.440 INFO:teuthology.orchestra.run.smithi187.stdout:5.15.0-56-generic 2024-04-25T00:41:51.440 INFO:teuthology.task.kernel:Running kernel on smithi187: 5.15.0-56-generic 2024-04-25T00:41:51.441 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get clean 2024-04-25T00:41:51.561 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get update 2024-04-25T00:41:51.569 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2024-04-25T00:41:51.696 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:41:51.706 INFO:teuthology.orchestra.run.smithi112.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:41:51.706 INFO:teuthology.orchestra.run.smithi112.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:41:51.716 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:41:51.730 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:41:51.792 INFO:teuthology.orchestra.run.smithi112.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T00:41:51.810 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T00:41:51.920 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T00:41:51.928 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T00:41:51.994 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T00:41:52.005 INFO:teuthology.orchestra.run.smithi112.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T00:41:52.005 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:41:52.038 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T00:41:52.048 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T00:41:52.053 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T00:41:52.054 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T00:41:52.054 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T00:41:52.055 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T00:41:52.062 INFO:teuthology.orchestra.run.smithi112.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T00:41:52.062 INFO:teuthology.orchestra.run.smithi112.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T00:41:52.095 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T00:41:52.100 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:41:52.101 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T00:41:52.111 INFO:teuthology.orchestra.run.smithi112.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T00:41:52.111 INFO:teuthology.orchestra.run.smithi112.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T00:41:52.118 INFO:teuthology.orchestra.run.smithi112.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:41:52.118 INFO:teuthology.orchestra.run.smithi112.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T00:41:52.127 INFO:teuthology.orchestra.run.smithi112.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T00:41:52.131 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T00:41:52.142 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T00:41:52.146 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T00:41:52.147 INFO:teuthology.orchestra.run.smithi112.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T00:41:52.148 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T00:41:52.149 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T00:41:52.150 INFO:teuthology.orchestra.run.smithi112.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T00:41:52.171 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T00:41:52.172 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T00:41:52.172 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T00:41:52.174 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T00:41:52.175 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T00:41:52.175 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T00:41:52.176 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T00:41:52.176 INFO:teuthology.orchestra.run.smithi112.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T00:41:52.176 INFO:teuthology.orchestra.run.smithi112.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T00:41:52.176 INFO:teuthology.orchestra.run.smithi112.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T00:41:52.176 INFO:teuthology.orchestra.run.smithi112.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T00:41:52.177 INFO:teuthology.orchestra.run.smithi112.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T00:41:52.177 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T00:41:52.186 INFO:teuthology.orchestra.run.smithi112.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T00:41:52.191 INFO:teuthology.orchestra.run.smithi112.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T00:41:52.192 INFO:teuthology.orchestra.run.smithi112.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T00:41:52.192 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T00:41:52.214 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T00:41:52.224 INFO:teuthology.orchestra.run.smithi112.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T00:41:52.225 INFO:teuthology.orchestra.run.smithi112.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T00:41:52.231 INFO:teuthology.orchestra.run.smithi112.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:41:52.231 INFO:teuthology.orchestra.run.smithi112.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T00:41:52.244 INFO:teuthology.orchestra.run.smithi112.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T00:41:52.252 INFO:teuthology.orchestra.run.smithi112.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T00:41:52.255 INFO:teuthology.orchestra.run.smithi112.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T00:41:52.255 INFO:teuthology.orchestra.run.smithi112.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T00:41:52.255 INFO:teuthology.orchestra.run.smithi112.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T00:41:52.256 INFO:teuthology.orchestra.run.smithi112.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T00:41:52.256 INFO:teuthology.orchestra.run.smithi112.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T00:41:52.275 INFO:teuthology.orchestra.run.smithi112.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T00:41:52.277 INFO:teuthology.orchestra.run.smithi112.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T00:41:52.278 INFO:teuthology.orchestra.run.smithi112.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T00:41:52.278 INFO:teuthology.orchestra.run.smithi112.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T00:41:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T00:41:52.296 INFO:teuthology.orchestra.run.smithi112.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T00:41:52.297 INFO:teuthology.orchestra.run.smithi112.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T00:41:52.297 INFO:teuthology.orchestra.run.smithi112.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T00:41:52.441 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T00:41:52.856 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T00:41:52.874 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T00:41:52.883 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T00:41:52.884 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T00:41:52.885 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T00:41:52.981 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T00:41:52.992 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:41:52.992 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T00:41:53.020 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T00:41:53.195 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T00:41:53.602 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T00:41:53.608 INFO:teuthology.orchestra.run.smithi187.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T00:41:53.622 INFO:teuthology.orchestra.run.smithi187.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T00:41:53.622 INFO:teuthology.orchestra.run.smithi187.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T00:41:53.625 INFO:teuthology.orchestra.run.smithi187.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T00:41:57.857 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 13.3 MB in 2s (7,273 kB/s) 2024-04-25T00:41:58.105 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 13.3 MB in 2s (6,700 kB/s) 2024-04-25T00:41:59.108 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2024-04-25T00:41:59.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:41:59.187 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2024-04-25T00:41:59.388 INFO:teuthology.orchestra.run.smithi112.stdout:Building dependency tree... 2024-04-25T00:41:59.388 INFO:teuthology.orchestra.run.smithi112.stdout:Reading state information... 2024-04-25T00:41:59.396 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T00:41:59.414 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:41:59.465 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi112.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:41:59.566 INFO:teuthology.orchestra.run.smithi112.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi112.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi112.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi112.stdout: python2.7-minimal 2024-04-25T00:41:59.567 INFO:teuthology.orchestra.run.smithi112.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi112.stdout:The following additional packages will be installed: 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi112.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi112.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T00:41:59.568 INFO:teuthology.orchestra.run.smithi112.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T00:41:59.570 INFO:teuthology.orchestra.run.smithi112.stdout:Suggested packages: 2024-04-25T00:41:59.570 INFO:teuthology.orchestra.run.smithi112.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T00:41:59.570 INFO:teuthology.orchestra.run.smithi112.stdout:Recommended packages: 2024-04-25T00:41:59.570 INFO:teuthology.orchestra.run.smithi112.stdout: thermald 2024-04-25T00:41:59.626 INFO:teuthology.orchestra.run.smithi112.stdout:The following NEW packages will be installed: 2024-04-25T00:41:59.626 INFO:teuthology.orchestra.run.smithi112.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:41:59.626 INFO:teuthology.orchestra.run.smithi112.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T00:41:59.627 INFO:teuthology.orchestra.run.smithi112.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T00:41:59.628 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages will be upgraded: 2024-04-25T00:41:59.628 INFO:teuthology.orchestra.run.smithi112.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T00:41:59.643 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2024-04-25T00:41:59.644 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2024-04-25T00:41:59.761 INFO:teuthology.orchestra.run.smithi112.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:41:59.761 INFO:teuthology.orchestra.run.smithi112.stdout:Need to get 113 MB of archives. 2024-04-25T00:41:59.761 INFO:teuthology.orchestra.run.smithi112.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T00:41:59.761 INFO:teuthology.orchestra.run.smithi112.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T00:41:59.798 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:41:59.798 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:41:59.798 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:41:59.799 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:41:59.799 INFO:teuthology.orchestra.run.smithi187.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T00:41:59.799 INFO:teuthology.orchestra.run.smithi187.stdout: python2.7-minimal 2024-04-25T00:41:59.799 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:41:59.800 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2024-04-25T00:41:59.800 INFO:teuthology.orchestra.run.smithi187.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:41:59.800 INFO:teuthology.orchestra.run.smithi187.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T00:41:59.800 INFO:teuthology.orchestra.run.smithi187.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T00:41:59.801 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2024-04-25T00:41:59.801 INFO:teuthology.orchestra.run.smithi187.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T00:41:59.801 INFO:teuthology.orchestra.run.smithi187.stdout:Recommended packages: 2024-04-25T00:41:59.801 INFO:teuthology.orchestra.run.smithi187.stdout: thermald 2024-04-25T00:41:59.845 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2024-04-25T00:41:59.845 INFO:teuthology.orchestra.run.smithi187.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:41:59.845 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T00:41:59.845 INFO:teuthology.orchestra.run.smithi187.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T00:41:59.846 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages will be upgraded: 2024-04-25T00:41:59.847 INFO:teuthology.orchestra.run.smithi187.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T00:41:59.945 INFO:teuthology.orchestra.run.smithi187.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:41:59.946 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 113 MB of archives. 2024-04-25T00:41:59.946 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T00:41:59.946 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T00:42:00.562 INFO:teuthology.orchestra.run.smithi112.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T00:42:00.823 INFO:teuthology.orchestra.run.smithi112.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T00:42:00.897 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T00:42:01.123 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T00:42:02.411 INFO:teuthology.orchestra.run.smithi112.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T00:42:02.411 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T00:42:02.411 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T00:42:02.689 INFO:teuthology.orchestra.run.smithi112.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T00:42:02.729 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T00:42:02.882 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T00:42:02.882 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T00:42:02.882 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T00:42:03.042 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T00:42:03.072 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T00:42:03.122 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 113 MB in 3s (36.6 MB/s) 2024-04-25T00:42:03.286 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T00:42:03.466 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 113 MB in 3s (35.2 MB/s) 2024-04-25T00:42:03.648 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T00:42:05.248 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T00:42:05.252 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:42:05.333 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:05.695 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T00:42:05.699 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:42:05.778 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:07.360 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T00:42:07.381 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:42:07.475 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:07.930 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T00:42:07.950 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:42:07.988 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:08.029 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T00:42:08.050 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:42:08.126 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:08.976 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T00:42:08.997 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:42:09.034 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:14.200 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:42:14.325 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:42:14.607 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:42:14.703 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:42:14.964 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T00:42:14.987 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T00:42:15.030 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:42:15.439 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:42:15.564 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:42:15.906 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:42:16.026 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:42:16.296 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T00:42:16.318 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T00:42:16.362 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:42:20.596 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T00:42:20.634 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:42:20.670 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:21.766 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T00:42:21.802 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:42:21.840 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:23.276 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:42:23.399 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:42:23.723 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:42:23.836 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:23.962 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T00:42:24.091 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:24.959 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:42:25.084 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:42:25.425 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:42:25.436 INFO:teuthology.orchestra.run.smithi112.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:42:25.436 INFO:teuthology.orchestra.run.smithi112.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T00:42:25.436 INFO:teuthology.orchestra.run.smithi112.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:42:25.436 INFO:teuthology.orchestra.run.smithi112.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T00:42:25.547 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:25.601 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:25.673 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T00:42:25.801 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:27.019 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:27.141 INFO:teuthology.orchestra.run.smithi187.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:42:27.141 INFO:teuthology.orchestra.run.smithi187.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T00:42:27.141 INFO:teuthology.orchestra.run.smithi187.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:42:27.141 INFO:teuthology.orchestra.run.smithi187.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T00:42:28.029 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:28.403 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T00:42:28.529 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T00:42:28.655 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:28.713 INFO:teuthology.orchestra.run.smithi112.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T00:42:28.713 INFO:teuthology.orchestra.run.smithi112.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:42:29.423 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:30.831 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T00:42:30.957 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T00:42:31.084 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:42:31.156 INFO:teuthology.orchestra.run.smithi187.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T00:42:31.156 INFO:teuthology.orchestra.run.smithi187.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:42:43.453 INFO:teuthology.orchestra.run.smithi112.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T00:42:43.453 INFO:teuthology.orchestra.run.smithi112.stdout:Sourcing file `/etc/default/grub' 2024-04-25T00:42:43.477 INFO:teuthology.orchestra.run.smithi112.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:42:43.503 INFO:teuthology.orchestra.run.smithi112.stdout:Generating grub configuration file ... 2024-04-25T00:42:43.747 INFO:teuthology.orchestra.run.smithi112.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:42:43.771 INFO:teuthology.orchestra.run.smithi112.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:42:44.075 INFO:teuthology.orchestra.run.smithi112.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:42:44.081 INFO:teuthology.orchestra.run.smithi112.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:42:44.148 INFO:teuthology.orchestra.run.smithi112.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:42:44.153 INFO:teuthology.orchestra.run.smithi112.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:42:44.312 INFO:teuthology.orchestra.run.smithi112.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:42:44.312 INFO:teuthology.orchestra.run.smithi112.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:42:44.312 INFO:teuthology.orchestra.run.smithi112.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:42:44.361 INFO:teuthology.orchestra.run.smithi112.stdout:done 2024-04-25T00:42:44.832 DEBUG:teuthology.orchestra.run.smithi112:> dpkg -s linux-image-generic 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Package: linux-image-generic 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Status: install ok installed 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Priority: optional 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Section: kernel 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Installed-Size: 21 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: amd64 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Source: linux-meta 2024-04-25T00:42:44.849 INFO:teuthology.orchestra.run.smithi112.stdout:Version: 5.15.0.105.102 2024-04-25T00:42:44.850 INFO:teuthology.orchestra.run.smithi112.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:42:44.850 INFO:teuthology.orchestra.run.smithi112.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:42:44.850 INFO:teuthology.orchestra.run.smithi112.stdout:Recommends: thermald 2024-04-25T00:42:44.850 INFO:teuthology.orchestra.run.smithi112.stdout:Description: Generic Linux kernel image 2024-04-25T00:42:44.850 INFO:teuthology.orchestra.run.smithi112.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:42:44.850 INFO:teuthology.orchestra.run.smithi112.stdout: available. 2024-04-25T00:42:44.850 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T00:42:44.850 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T00:42:44.850 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T00:42:44.850 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T00:42:44.851 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T00:42:44.851 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get clean 2024-04-25T00:42:44.990 INFO:teuthology.orchestra.run.smithi187.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T00:42:44.990 INFO:teuthology.orchestra.run.smithi187.stdout:Sourcing file `/etc/default/grub' 2024-04-25T00:42:45.008 INFO:teuthology.orchestra.run.smithi187.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:42:45.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get update 2024-04-25T00:42:45.037 INFO:teuthology.orchestra.run.smithi187.stdout:Generating grub configuration file ... 2024-04-25T00:42:45.262 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:42:45.262 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:42:45.324 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:42:45.342 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:42:45.357 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:42:45.406 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:42:45.731 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:42:45.736 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:42:45.806 INFO:teuthology.orchestra.run.smithi187.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:42:45.812 INFO:teuthology.orchestra.run.smithi187.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:42:45.959 INFO:teuthology.orchestra.run.smithi187.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:42:45.959 INFO:teuthology.orchestra.run.smithi187.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:42:45.960 INFO:teuthology.orchestra.run.smithi187.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:42:45.996 INFO:teuthology.orchestra.run.smithi187.stdout:done 2024-04-25T00:42:46.453 DEBUG:teuthology.orchestra.run.smithi187:> dpkg -s linux-image-generic 2024-04-25T00:42:46.471 INFO:teuthology.orchestra.run.smithi187.stdout:Package: linux-image-generic 2024-04-25T00:42:46.471 INFO:teuthology.orchestra.run.smithi187.stdout:Status: install ok installed 2024-04-25T00:42:46.471 INFO:teuthology.orchestra.run.smithi187.stdout:Priority: optional 2024-04-25T00:42:46.471 INFO:teuthology.orchestra.run.smithi187.stdout:Section: kernel 2024-04-25T00:42:46.471 INFO:teuthology.orchestra.run.smithi187.stdout:Installed-Size: 21 2024-04-25T00:42:46.471 INFO:teuthology.orchestra.run.smithi187.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:42:46.471 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: amd64 2024-04-25T00:42:46.472 INFO:teuthology.orchestra.run.smithi187.stdout:Source: linux-meta 2024-04-25T00:42:46.472 INFO:teuthology.orchestra.run.smithi187.stdout:Version: 5.15.0.105.102 2024-04-25T00:42:46.472 INFO:teuthology.orchestra.run.smithi187.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:42:46.472 INFO:teuthology.orchestra.run.smithi187.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:42:46.472 INFO:teuthology.orchestra.run.smithi187.stdout:Recommends: thermald 2024-04-25T00:42:46.472 INFO:teuthology.orchestra.run.smithi187.stdout:Description: Generic Linux kernel image 2024-04-25T00:42:46.472 INFO:teuthology.orchestra.run.smithi187.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:42:46.472 INFO:teuthology.orchestra.run.smithi187.stdout: available. 2024-04-25T00:42:46.472 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T00:42:46.472 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T00:42:46.472 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T00:42:46.473 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T00:42:46.473 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T00:42:46.473 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get clean 2024-04-25T00:42:46.629 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2024-04-25T00:42:46.646 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2024-04-25T00:42:46.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:42:46.706 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2024-04-25T00:42:46.765 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:42:46.777 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:42:46.797 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:42:46.882 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:42:46.907 INFO:teuthology.orchestra.run.smithi112.stdout:Building dependency tree... 2024-04-25T00:42:46.907 INFO:teuthology.orchestra.run.smithi112.stdout:Reading state information... 2024-04-25T00:42:47.089 INFO:teuthology.orchestra.run.smithi112.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T00:42:47.089 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:42:47.089 INFO:teuthology.orchestra.run.smithi112.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:42:47.089 INFO:teuthology.orchestra.run.smithi112.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:42:47.089 INFO:teuthology.orchestra.run.smithi112.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:42:47.090 INFO:teuthology.orchestra.run.smithi112.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:42:47.090 INFO:teuthology.orchestra.run.smithi112.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:42:47.090 INFO:teuthology.orchestra.run.smithi112.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:42:47.132 INFO:teuthology.orchestra.run.smithi112.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:42:47.133 DEBUG:teuthology.orchestra.run.smithi112:> dpkg -s linux-image-generic 2024-04-25T00:42:47.146 INFO:teuthology.orchestra.run.smithi112.stdout:Package: linux-image-generic 2024-04-25T00:42:47.146 INFO:teuthology.orchestra.run.smithi112.stdout:Status: install ok installed 2024-04-25T00:42:47.146 INFO:teuthology.orchestra.run.smithi112.stdout:Priority: optional 2024-04-25T00:42:47.146 INFO:teuthology.orchestra.run.smithi112.stdout:Section: kernel 2024-04-25T00:42:47.146 INFO:teuthology.orchestra.run.smithi112.stdout:Installed-Size: 21 2024-04-25T00:42:47.146 INFO:teuthology.orchestra.run.smithi112.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:42:47.146 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: amd64 2024-04-25T00:42:47.146 INFO:teuthology.orchestra.run.smithi112.stdout:Source: linux-meta 2024-04-25T00:42:47.147 INFO:teuthology.orchestra.run.smithi112.stdout:Version: 5.15.0.105.102 2024-04-25T00:42:47.147 INFO:teuthology.orchestra.run.smithi112.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:42:47.147 INFO:teuthology.orchestra.run.smithi112.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:42:47.147 INFO:teuthology.orchestra.run.smithi112.stdout:Recommends: thermald 2024-04-25T00:42:47.147 INFO:teuthology.orchestra.run.smithi112.stdout:Description: Generic Linux kernel image 2024-04-25T00:42:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:42:47.147 INFO:teuthology.orchestra.run.smithi112.stdout: available. 2024-04-25T00:42:47.147 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-04-25T00:42:47.194 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.w5GxrheK1r 2024-04-25T00:42:47.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub/grub.cfg /tmp/tmp.w5GxrheK1r 2024-04-25T00:42:47.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.w5GxrheK1r 2024-04-25T00:42:47.369 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.w5GxrheK1r is 10KB 2024-04-25T00:42:47.382 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.w5GxrheK1r 2024-04-25T00:42:47.387 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T00:42:47.445 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:42:47.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T00:42:47.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T00:42:47.516 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T00:42:47.517 DEBUG:teuthology.orchestra.run.smithi112:> sudo update-grub 2024-04-25T00:42:48.140 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T00:42:48.156 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:42:48.207 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T00:42:48.384 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2024-04-25T00:42:48.384 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2024-04-25T00:42:48.543 INFO:teuthology.orchestra.run.smithi187.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T00:42:48.544 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:42:48.544 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:42:48.544 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:42:48.544 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:42:48.544 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:42:48.544 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:42:48.544 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:42:48.583 INFO:teuthology.orchestra.run.smithi187.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:42:48.585 DEBUG:teuthology.orchestra.run.smithi187:> dpkg -s linux-image-generic 2024-04-25T00:42:48.641 INFO:teuthology.orchestra.run.smithi187.stdout:Package: linux-image-generic 2024-04-25T00:42:48.641 INFO:teuthology.orchestra.run.smithi187.stdout:Status: install ok installed 2024-04-25T00:42:48.641 INFO:teuthology.orchestra.run.smithi187.stdout:Priority: optional 2024-04-25T00:42:48.641 INFO:teuthology.orchestra.run.smithi187.stdout:Section: kernel 2024-04-25T00:42:48.641 INFO:teuthology.orchestra.run.smithi187.stdout:Installed-Size: 21 2024-04-25T00:42:48.641 INFO:teuthology.orchestra.run.smithi187.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:42:48.641 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: amd64 2024-04-25T00:42:48.642 INFO:teuthology.orchestra.run.smithi187.stdout:Source: linux-meta 2024-04-25T00:42:48.642 INFO:teuthology.orchestra.run.smithi187.stdout:Version: 5.15.0.105.102 2024-04-25T00:42:48.642 INFO:teuthology.orchestra.run.smithi187.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:42:48.642 INFO:teuthology.orchestra.run.smithi187.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:42:48.642 INFO:teuthology.orchestra.run.smithi187.stdout:Recommends: thermald 2024-04-25T00:42:48.642 INFO:teuthology.orchestra.run.smithi187.stdout:Description: Generic Linux kernel image 2024-04-25T00:42:48.642 INFO:teuthology.orchestra.run.smithi187.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:42:48.642 INFO:teuthology.orchestra.run.smithi187.stdout: available. 2024-04-25T00:42:48.642 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-25T00:42:48.651 INFO:teuthology.orchestra.run.smithi112.stderr:Sourcing file `/etc/default/grub' 2024-04-25T00:42:48.652 INFO:teuthology.orchestra.run.smithi112.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:42:48.654 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2024-04-25T00:42:48.687 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.Ai79KIca1Z 2024-04-25T00:42:48.688 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Ai79KIca1Z 2024-04-25T00:42:48.738 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.Ai79KIca1Z 2024-04-25T00:42:48.843 INFO:teuthology.orchestra.run.smithi112.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:42:48.849 INFO:teuthology.orchestra.run.smithi112.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:42:48.855 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.Ai79KIca1Z is 10KB 2024-04-25T00:42:48.869 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.Ai79KIca1Z 2024-04-25T00:42:48.874 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T00:42:48.926 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:42:48.926 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T00:42:48.926 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T00:42:48.990 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T00:42:48.990 DEBUG:teuthology.orchestra.run.smithi187:> sudo update-grub 2024-04-25T00:42:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:42:49.086 INFO:teuthology.orchestra.run.smithi112.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:42:49.127 INFO:teuthology.orchestra.run.smithi112.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:42:49.132 INFO:teuthology.orchestra.run.smithi112.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:42:49.249 INFO:teuthology.orchestra.run.smithi112.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:42:49.249 INFO:teuthology.orchestra.run.smithi112.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:42:49.249 INFO:teuthology.orchestra.run.smithi112.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:42:49.270 INFO:teuthology.orchestra.run.smithi112.stderr:done 2024-04-25T00:42:49.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2024-04-25T00:42:50.201 INFO:teuthology.orchestra.run.smithi187.stderr:Sourcing file `/etc/default/grub' 2024-04-25T00:42:50.202 INFO:teuthology.orchestra.run.smithi187.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:42:50.205 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2024-04-25T00:42:50.461 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:42:50.467 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:42:50.785 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:42:50.790 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:42:50.832 INFO:teuthology.orchestra.run.smithi187.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:42:50.838 INFO:teuthology.orchestra.run.smithi187.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:42:50.962 INFO:teuthology.orchestra.run.smithi187.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:42:50.962 INFO:teuthology.orchestra.run.smithi187.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:42:50.962 INFO:teuthology.orchestra.run.smithi187.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:42:50.980 INFO:teuthology.orchestra.run.smithi187.stderr:done 2024-04-25T00:42:50.989 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2024-04-25T00:43:19.315 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T00:43:19.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T00:43:19.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:43:20.993 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T00:43:20.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T00:43:20.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:44:19.375 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T00:44:20.998 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T00:44:28.384 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T00:44:28.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:44:28.388 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2024-04-25T00:44:30.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T00:44:30.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:44:30.680 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-25T00:44:31.468 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T00:44:31.468 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T00:44:31.469 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2024-04-25T00:44:31.515 INFO:teuthology.orchestra.run.smithi187.stdout:5.15.0-105-generic 2024-04-25T00:44:31.515 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T00:44:31.515 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:44:31.516 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T00:44:32.516 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T00:44:32.516 DEBUG:teuthology.orchestra.run.smithi187:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:44:32.696 INFO:teuthology.orchestra.run.smithi187.stdout:ttyS1 2024-04-25T00:44:32.713 DEBUG:teuthology.parallel:result is None 2024-04-25T00:44:40.389 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T00:44:40.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:44:40.657 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T00:44:41.140 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T00:44:41.140 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T00:44:41.140 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2024-04-25T00:44:41.187 INFO:teuthology.orchestra.run.smithi112.stdout:5.15.0-105-generic 2024-04-25T00:44:41.187 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T00:44:41.187 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:44:41.188 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T00:44:42.188 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T00:44:42.189 DEBUG:teuthology.orchestra.run.smithi112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:44:42.273 INFO:teuthology.orchestra.run.smithi112.stdout:ttyS1 2024-04-25T00:44:42.304 DEBUG:teuthology.parallel:result is None 2024-04-25T00:44:42.305 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T00:44:42.314 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T00:44:42.315 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:44:42.317 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:44:42.323 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T00:44:42.330 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T00:44:42.336 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T00:44:42.337 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:44:42.365 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:44:42.467 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T00:44:42.473 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T00:44:42.474 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:44:42.476 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:44:42.494 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:44:42.496 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:44:42.504 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:44:42.504 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:44:42.506 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T00:44:42.514 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T00:44:42.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:44:42.556 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:44:42.570 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T00:44:42.579 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T00:44:42.579 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:44:42.612 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:44:42.618 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:44:42.663 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:44:42.714 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:44:42.714 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:44:42.776 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:44:42.781 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:44:42.828 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:44:42.828 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:44:42.893 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2024-04-25T00:44:42.896 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2024-04-25T00:44:42.961 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T00:44:42.969 INFO:teuthology.task.internal:Starting timer... 2024-04-25T00:44:42.969 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T00:44:42.979 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T00:44:42.988 DEBUG:teuthology.task.selinux:Excluding smithi112: OS 'ubuntu' does not support SELinux 2024-04-25T00:44:42.988 DEBUG:teuthology.task.selinux:Excluding smithi187: OS 'ubuntu' does not support SELinux 2024-04-25T00:44:42.988 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T00:44:42.988 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T00:44:42.988 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T00:44:42.988 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T00:44:42.997 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T00:44:43.015 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T00:44:43.017 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi112.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:51:43.142 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi112.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2024-04-25T00:51:43.143 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T00:51:43.145 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:51:43.229 DEBUG:teuthology.orchestra.run.smithi112:> true 2024-04-25T00:51:43.307 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2024-04-25T00:51:43.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T00:51:43.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:51:43.389 DEBUG:teuthology.orchestra.run.smithi187:> true 2024-04-25T00:51:43.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2024-04-25T00:51:43.466 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:51:43.480 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:51:43.480 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:51:43.481 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:51:43.483 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:51:43.483 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:51:43.511 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T00:51:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Command line: ntpd -gq 2024-04-25T00:51:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: ---------------------------------------------------- 2024-04-25T00:51:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T00:51:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T00:51:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: corporation. Support and training for ntp-4 are 2024-04-25T00:51:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: available at https://www.nwtime.org/support 2024-04-25T00:51:43.512 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: ---------------------------------------------------- 2024-04-25T00:51:43.513 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: proto: precision = 0.062 usec (-24) 2024-04-25T00:51:43.513 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: basedate set to 2022-02-04 2024-04-25T00:51:43.513 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: gps base set to 2022-02-06 (week 2196) 2024-04-25T00:51:43.513 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:51:43.514 INFO:teuthology.orchestra.run.smithi112.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:51:43.514 INFO:teuthology.orchestra.run.smithi112.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:51:43.515 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:51:43.515 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T00:51:43.515 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T00:51:43.515 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T00:51:43.515 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Listen normally on 3 enp3s0f1 172.21.15.112:123 2024-04-25T00:51:43.515 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Listen normally on 4 lo [::1]:123 2024-04-25T00:51:43.516 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cb69%5]:123 2024-04-25T00:51:43.516 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:43 ntpd[17231]: Listening on routing socket on fd #22 for interface updates 2024-04-25T00:51:43.535 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T00:51:43.535 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Command line: ntpd -gq 2024-04-25T00:51:43.535 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: ---------------------------------------------------- 2024-04-25T00:51:43.535 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: corporation. Support and training for ntp-4 are 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: available at https://www.nwtime.org/support 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: ---------------------------------------------------- 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: proto: precision = 0.043 usec (-24) 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: basedate set to 2022-02-04 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: gps base set to 2022-02-06 (week 2196) 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T00:51:43.536 INFO:teuthology.orchestra.run.smithi187.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:51:43.538 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T00:51:43.538 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T00:51:43.538 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T00:51:43.538 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T00:51:43.538 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Listen normally on 3 enp3s0f1 172.21.15.187:123 2024-04-25T00:51:43.538 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Listen normally on 4 lo [::1]:123 2024-04-25T00:51:43.538 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:77b1%5]:123 2024-04-25T00:51:43.538 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:43 ntpd[17199]: Listening on routing socket on fd #22 for interface updates 2024-04-25T00:51:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:44 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:44 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:44 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:44 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:44 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:44 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:44.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:44.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:44.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:44.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:44.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:44.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:44.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:44.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:44.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:44.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:44.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:44.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:44 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:45.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:45 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:45.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:45 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:45.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:45 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:45.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:45 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:45.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:45 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:45.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:45 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:46 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:46 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:46 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:46 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:46 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:46 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:46 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:46.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:46 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:46.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:46.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:46.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:46 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:47.512 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:47 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:47.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:47 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:47.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:47 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:47.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:47 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:47.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:47 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:47.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:47 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:47.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:47 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:47.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:47 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:47.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:47 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:47.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:47 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:47.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:47 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:47.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:47 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:47.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:47 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:47.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:47 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:48.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:48.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:48.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:48.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:48.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:48.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:48.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:48.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:48.515 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:48.515 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:48.515 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:48.515 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:48 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:48.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:48.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:48.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:48.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:48.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:48.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:48.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:48.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:48.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:48.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:48.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:48.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:48 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:49.512 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:49 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:49.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:49 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:49.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:49 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:49.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:49 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:49.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:49 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:49.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:49 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:49.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:49 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:49.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:49 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:49.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:49 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:49.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:49 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:49.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:49 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:49.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:49 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:50.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:50.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:50.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:50.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:50.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:50.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:50.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:50.513 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:50.514 INFO:teuthology.orchestra.run.smithi112.stdout:25 Apr 00:51:50 ntpd[17231]: ntpd: time slew -0.001894 s 2024-04-25T00:51:50.514 INFO:teuthology.orchestra.run.smithi112.stdout:ntpd: time slew -0.001894s 2024-04-25T00:51:50.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T00:51:50.514 INFO:teuthology.orchestra.run.smithi112.stderr:25 Apr 00:51:50 ntpd[17231]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T00:51:50.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:50.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:50.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:50.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:50.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T00:51:50.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T00:51:50.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T00:51:50.535 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T00:51:50.536 INFO:teuthology.orchestra.run.smithi187.stdout:25 Apr 00:51:50 ntpd[17199]: ntpd: time slew -0.000908 s 2024-04-25T00:51:50.536 INFO:teuthology.orchestra.run.smithi187.stdout:ntpd: time slew -0.000908s 2024-04-25T00:51:50.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T00:51:50.536 INFO:teuthology.orchestra.run.smithi187.stderr:25 Apr 00:51:50 ntpd[17199]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T00:51:50.584 INFO:teuthology.orchestra.run.smithi187.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:51:50.584 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================== 2024-04-25T00:51:50.585 INFO:teuthology.orchestra.run.smithi187.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:51:50.585 INFO:teuthology.orchestra.run.smithi187.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:51:50.585 INFO:teuthology.orchestra.run.smithi187.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:51:50.585 INFO:teuthology.orchestra.run.smithi187.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:51:50.653 INFO:teuthology.orchestra.run.smithi112.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:51:50.653 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================== 2024-04-25T00:51:50.654 INFO:teuthology.orchestra.run.smithi112.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:51:50.654 INFO:teuthology.orchestra.run.smithi112.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:51:50.654 INFO:teuthology.orchestra.run.smithi112.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:51:50.654 INFO:teuthology.orchestra.run.smithi112.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:51:50.655 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:51:50.665 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:51:50.665 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:51:50.666 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:51:50.666 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:51:50.671 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:51:50.671 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:51:50.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-key list | grep Ceph 2024-04-25T00:51:50.672 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-key list | grep Ceph 2024-04-25T00:51:50.735 INFO:teuthology.orchestra.run.smithi187.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T00:51:50.758 INFO:teuthology.orchestra.run.smithi112.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T00:51:50.762 INFO:teuthology.orchestra.run.smithi187.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T00:51:50.762 INFO:teuthology.orchestra.run.smithi187.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T00:51:50.763 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T00:51:50.763 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T00:51:50.764 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:51:50.784 INFO:teuthology.orchestra.run.smithi112.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T00:51:50.784 INFO:teuthology.orchestra.run.smithi112.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T00:51:50.784 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T00:51:50.784 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T00:51:50.784 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:51:50.904 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T00:51:50.904 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:51:50.925 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T00:51:50.925 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:51:51.282 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:51:51.283 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T00:51:51.297 DEBUG:teuthology.orchestra.run.smithi187:> sudo apt-get update 2024-04-25T00:51:51.303 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:51:51.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T00:51:51.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo apt-get update 2024-04-25T00:51:51.492 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:51:51.494 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:51:51.494 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:51:51.505 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:51:51.507 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:51:51.525 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:51:51.528 INFO:teuthology.orchestra.run.smithi112.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:51:51.586 INFO:teuthology.orchestra.run.smithi187.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:51:51.752 INFO:teuthology.orchestra.run.smithi112.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T00:51:51.795 INFO:teuthology.orchestra.run.smithi187.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T00:51:51.827 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T00:51:51.887 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T00:51:51.905 INFO:teuthology.orchestra.run.smithi112.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T00:51:51.979 INFO:teuthology.orchestra.run.smithi187.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T00:51:51.981 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T00:51:52.071 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T00:51:52.142 INFO:teuthology.orchestra.run.smithi112.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T00:51:52.176 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T00:51:52.228 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 28.9 kB in 1s (40.4 kB/s) 2024-04-25T00:51:52.274 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 28.9 kB in 1s (37.2 kB/s) 2024-04-25T00:51:53.205 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2024-04-25T00:51:53.220 DEBUG:teuthology.orchestra.run.smithi112:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:51:53.278 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T00:51:53.281 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2024-04-25T00:51:53.295 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:51:53.356 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T00:51:53.477 INFO:teuthology.orchestra.run.smithi112.stdout:Building dependency tree... 2024-04-25T00:51:53.477 INFO:teuthology.orchestra.run.smithi112.stdout:Reading state information... 2024-04-25T00:51:53.555 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2024-04-25T00:51:53.555 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2024-04-25T00:51:53.674 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:51:53.674 INFO:teuthology.orchestra.run.smithi112.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:51:53.674 INFO:teuthology.orchestra.run.smithi112.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:51:53.674 INFO:teuthology.orchestra.run.smithi112.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:51:53.675 INFO:teuthology.orchestra.run.smithi112.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:51:53.675 INFO:teuthology.orchestra.run.smithi112.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:51:53.675 INFO:teuthology.orchestra.run.smithi112.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:51:53.675 INFO:teuthology.orchestra.run.smithi112.stdout:The following additional packages will be installed: 2024-04-25T00:51:53.675 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T00:51:53.677 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T00:51:53.678 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T00:51:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T00:51:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T00:51:53.679 INFO:teuthology.orchestra.run.smithi112.stdout:Suggested packages: 2024-04-25T00:51:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T00:51:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T00:51:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T00:51:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T00:51:53.679 INFO:teuthology.orchestra.run.smithi112.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T00:51:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T00:51:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T00:51:53.680 INFO:teuthology.orchestra.run.smithi112.stdout:Recommended packages: 2024-04-25T00:51:53.680 INFO:teuthology.orchestra.run.smithi112.stdout: btrfs-tools 2024-04-25T00:51:53.753 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:51:53.753 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:51:53.753 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:51:53.753 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:51:53.754 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:51:53.755 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:51:53.755 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:51:53.755 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2024-04-25T00:51:53.755 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T00:51:53.755 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T00:51:53.755 INFO:teuthology.orchestra.run.smithi187.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:51:53.755 INFO:teuthology.orchestra.run.smithi187.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T00:51:53.756 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T00:51:53.756 INFO:teuthology.orchestra.run.smithi187.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T00:51:53.757 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T00:51:53.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T00:51:53.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T00:51:53.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T00:51:53.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T00:51:53.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T00:51:53.758 INFO:teuthology.orchestra.run.smithi187.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T00:51:53.758 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2024-04-25T00:51:53.758 INFO:teuthology.orchestra.run.smithi187.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T00:51:53.759 INFO:teuthology.orchestra.run.smithi187.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T00:51:53.759 INFO:teuthology.orchestra.run.smithi187.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T00:51:53.759 INFO:teuthology.orchestra.run.smithi187.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T00:51:53.759 INFO:teuthology.orchestra.run.smithi187.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T00:51:53.759 INFO:teuthology.orchestra.run.smithi187.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T00:51:53.759 INFO:teuthology.orchestra.run.smithi187.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T00:51:53.759 INFO:teuthology.orchestra.run.smithi187.stdout:Recommended packages: 2024-04-25T00:51:53.759 INFO:teuthology.orchestra.run.smithi187.stdout: btrfs-tools 2024-04-25T00:51:53.780 INFO:teuthology.orchestra.run.smithi112.stdout:The following NEW packages will be installed: 2024-04-25T00:51:53.780 INFO:teuthology.orchestra.run.smithi112.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T00:51:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T00:51:53.781 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T00:51:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T00:51:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:51:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T00:51:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T00:51:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T00:51:53.783 INFO:teuthology.orchestra.run.smithi112.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T00:51:53.784 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T00:51:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T00:51:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T00:51:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T00:51:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T00:51:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T00:51:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T00:51:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T00:51:53.785 INFO:teuthology.orchestra.run.smithi112.stdout: socat unzip xmlstarlet zip 2024-04-25T00:51:53.864 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T00:51:53.865 INFO:teuthology.orchestra.run.smithi187.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T00:51:53.866 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T00:51:53.867 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T00:51:53.867 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T00:51:53.867 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T00:51:53.867 INFO:teuthology.orchestra.run.smithi187.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T00:51:53.867 INFO:teuthology.orchestra.run.smithi187.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T00:51:53.867 INFO:teuthology.orchestra.run.smithi187.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T00:51:53.867 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T00:51:53.867 INFO:teuthology.orchestra.run.smithi187.stdout: socat unzip xmlstarlet zip 2024-04-25T00:51:53.969 INFO:teuthology.orchestra.run.smithi112.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T00:51:53.970 INFO:teuthology.orchestra.run.smithi112.stdout:Need to get 180 MB of archives. 2024-04-25T00:51:53.970 INFO:teuthology.orchestra.run.smithi112.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T00:51:53.970 INFO:teuthology.orchestra.run.smithi112.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T00:51:54.052 INFO:teuthology.orchestra.run.smithi112.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T00:51:54.055 INFO:teuthology.orchestra.run.smithi187.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T00:51:54.055 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 180 MB of archives. 2024-04-25T00:51:54.055 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T00:51:54.055 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T00:51:54.098 INFO:teuthology.orchestra.run.smithi112.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T00:51:54.138 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T00:51:54.183 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T00:51:54.279 INFO:teuthology.orchestra.run.smithi112.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T00:51:54.299 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T00:51:54.371 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T00:51:54.385 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T00:51:54.393 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T00:51:54.407 INFO:teuthology.orchestra.run.smithi112.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T00:51:54.458 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T00:51:54.480 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T00:51:54.494 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T00:51:54.546 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T00:51:54.632 INFO:teuthology.orchestra.run.smithi112.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T00:51:54.641 INFO:teuthology.orchestra.run.smithi112.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T00:51:54.668 INFO:teuthology.orchestra.run.smithi112.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T00:51:54.676 INFO:teuthology.orchestra.run.smithi112.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T00:51:54.733 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T00:51:54.741 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T00:51:54.765 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T00:51:54.773 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T00:51:54.841 INFO:teuthology.orchestra.run.smithi112.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T00:51:54.938 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T00:51:55.075 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T00:51:55.406 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T00:51:55.415 INFO:teuthology.orchestra.run.smithi112.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T00:51:55.430 INFO:teuthology.orchestra.run.smithi112.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T00:51:55.446 INFO:teuthology.orchestra.run.smithi112.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T00:51:55.455 INFO:teuthology.orchestra.run.smithi112.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T00:51:55.455 INFO:teuthology.orchestra.run.smithi112.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T00:51:55.471 INFO:teuthology.orchestra.run.smithi112.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T00:51:55.486 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T00:51:55.498 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T00:51:55.500 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T00:51:55.509 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T00:51:55.509 INFO:teuthology.orchestra.run.smithi112.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T00:51:55.522 INFO:teuthology.orchestra.run.smithi112.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T00:51:55.524 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T00:51:55.526 INFO:teuthology.orchestra.run.smithi112.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T00:51:55.526 INFO:teuthology.orchestra.run.smithi112.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T00:51:55.539 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T00:51:55.548 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T00:51:55.549 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T00:51:55.560 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T00:51:55.563 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T00:51:55.565 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T00:51:55.576 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T00:51:55.603 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T00:51:55.617 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T00:51:55.620 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T00:51:55.621 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T00:51:55.697 INFO:teuthology.orchestra.run.smithi112.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T00:51:55.780 INFO:teuthology.orchestra.run.smithi112.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T00:51:55.790 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T00:51:55.801 INFO:teuthology.orchestra.run.smithi112.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T00:51:55.861 INFO:teuthology.orchestra.run.smithi112.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T00:51:55.873 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T00:51:55.879 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T00:51:55.900 INFO:teuthology.orchestra.run.smithi112.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T00:51:55.922 INFO:teuthology.orchestra.run.smithi112.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T00:51:55.950 INFO:teuthology.orchestra.run.smithi112.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T00:51:55.954 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T00:51:55.980 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T00:51:55.996 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T00:51:56.015 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T00:51:56.133 INFO:teuthology.orchestra.run.smithi112.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T00:51:56.148 INFO:teuthology.orchestra.run.smithi112.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T00:51:56.156 INFO:teuthology.orchestra.run.smithi112.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T00:51:56.161 INFO:teuthology.orchestra.run.smithi112.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T00:51:56.195 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T00:51:56.212 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T00:51:56.219 INFO:teuthology.orchestra.run.smithi187.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T00:51:56.223 INFO:teuthology.orchestra.run.smithi187.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T00:51:56.279 INFO:teuthology.orchestra.run.smithi187.stdout:Get:43 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T00:51:56.284 INFO:teuthology.orchestra.run.smithi187.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T00:51:56.346 INFO:teuthology.orchestra.run.smithi112.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T00:51:56.352 INFO:teuthology.orchestra.run.smithi187.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T00:51:56.408 INFO:teuthology.orchestra.run.smithi187.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T00:51:56.428 INFO:teuthology.orchestra.run.smithi112.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T00:51:56.444 INFO:teuthology.orchestra.run.smithi112.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T00:51:56.491 INFO:teuthology.orchestra.run.smithi187.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T00:51:56.505 INFO:teuthology.orchestra.run.smithi187.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T00:51:56.630 INFO:teuthology.orchestra.run.smithi112.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T00:51:56.681 INFO:teuthology.orchestra.run.smithi112.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T00:51:56.684 INFO:teuthology.orchestra.run.smithi187.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T00:51:56.695 INFO:teuthology.orchestra.run.smithi112.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T00:51:56.730 INFO:teuthology.orchestra.run.smithi187.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T00:51:56.743 INFO:teuthology.orchestra.run.smithi187.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T00:51:56.853 INFO:teuthology.orchestra.run.smithi112.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T00:51:56.854 INFO:teuthology.orchestra.run.smithi112.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T00:51:56.859 INFO:teuthology.orchestra.run.smithi112.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T00:51:56.863 INFO:teuthology.orchestra.run.smithi112.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T00:51:56.878 INFO:teuthology.orchestra.run.smithi112.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T00:51:56.905 INFO:teuthology.orchestra.run.smithi187.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T00:51:56.907 INFO:teuthology.orchestra.run.smithi187.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T00:51:56.911 INFO:teuthology.orchestra.run.smithi187.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T00:51:56.916 INFO:teuthology.orchestra.run.smithi187.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T00:51:56.930 INFO:teuthology.orchestra.run.smithi187.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T00:51:57.065 INFO:teuthology.orchestra.run.smithi112.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T00:51:57.115 INFO:teuthology.orchestra.run.smithi187.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T00:51:57.319 INFO:teuthology.orchestra.run.smithi112.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T00:51:57.371 INFO:teuthology.orchestra.run.smithi187.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T00:51:57.431 INFO:teuthology.orchestra.run.smithi112.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T00:51:57.474 INFO:teuthology.orchestra.run.smithi112.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T00:51:57.485 INFO:teuthology.orchestra.run.smithi187.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T00:51:57.529 INFO:teuthology.orchestra.run.smithi187.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T00:51:57.557 INFO:teuthology.orchestra.run.smithi112.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T00:51:57.569 INFO:teuthology.orchestra.run.smithi112.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T00:51:57.583 INFO:teuthology.orchestra.run.smithi112.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T00:51:57.610 INFO:teuthology.orchestra.run.smithi187.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T00:51:57.622 INFO:teuthology.orchestra.run.smithi112.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T00:51:57.623 INFO:teuthology.orchestra.run.smithi187.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T00:51:57.636 INFO:teuthology.orchestra.run.smithi187.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T00:51:57.676 INFO:teuthology.orchestra.run.smithi187.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T00:51:57.789 INFO:teuthology.orchestra.run.smithi112.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T00:51:57.883 INFO:teuthology.orchestra.run.smithi112.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T00:51:57.884 INFO:teuthology.orchestra.run.smithi112.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T00:51:57.887 INFO:teuthology.orchestra.run.smithi112.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T00:51:57.936 INFO:teuthology.orchestra.run.smithi187.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T00:51:57.938 INFO:teuthology.orchestra.run.smithi187.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T00:51:57.941 INFO:teuthology.orchestra.run.smithi187.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T00:51:58.052 INFO:teuthology.orchestra.run.smithi112.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T00:51:58.105 INFO:teuthology.orchestra.run.smithi187.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T00:51:58.388 INFO:teuthology.orchestra.run.smithi112.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T00:51:58.437 INFO:teuthology.orchestra.run.smithi187.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T00:51:58.784 INFO:teuthology.orchestra.run.smithi112.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T00:51:58.794 INFO:teuthology.orchestra.run.smithi112.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T00:51:58.795 INFO:teuthology.orchestra.run.smithi112.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T00:51:58.833 INFO:teuthology.orchestra.run.smithi187.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T00:51:58.842 INFO:teuthology.orchestra.run.smithi187.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T00:51:58.842 INFO:teuthology.orchestra.run.smithi187.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T00:51:58.843 INFO:teuthology.orchestra.run.smithi187.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T00:51:58.863 INFO:teuthology.orchestra.run.smithi112.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T00:51:58.864 INFO:teuthology.orchestra.run.smithi112.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T00:51:58.865 INFO:teuthology.orchestra.run.smithi112.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T00:51:58.867 INFO:teuthology.orchestra.run.smithi112.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T00:51:58.868 INFO:teuthology.orchestra.run.smithi112.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T00:51:58.869 INFO:teuthology.orchestra.run.smithi112.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T00:51:58.909 INFO:teuthology.orchestra.run.smithi187.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T00:51:58.910 INFO:teuthology.orchestra.run.smithi187.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T00:51:58.912 INFO:teuthology.orchestra.run.smithi187.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T00:51:58.915 INFO:teuthology.orchestra.run.smithi187.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T00:51:58.916 INFO:teuthology.orchestra.run.smithi187.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T00:51:58.917 INFO:teuthology.orchestra.run.smithi187.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T00:51:59.034 INFO:teuthology.orchestra.run.smithi112.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T00:51:59.082 INFO:teuthology.orchestra.run.smithi187.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T00:51:59.186 INFO:teuthology.orchestra.run.smithi187.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T00:51:59.195 INFO:teuthology.orchestra.run.smithi187.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T00:51:59.222 INFO:teuthology.orchestra.run.smithi187.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T00:51:59.250 INFO:teuthology.orchestra.run.smithi112.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T00:51:59.283 INFO:teuthology.orchestra.run.smithi187.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T00:51:59.335 INFO:teuthology.orchestra.run.smithi187.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T00:51:59.566 INFO:teuthology.orchestra.run.smithi112.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T00:51:59.585 INFO:teuthology.orchestra.run.smithi187.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T00:51:59.610 INFO:teuthology.orchestra.run.smithi112.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T00:51:59.628 INFO:teuthology.orchestra.run.smithi187.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T00:51:59.637 INFO:teuthology.orchestra.run.smithi112.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T00:51:59.647 INFO:teuthology.orchestra.run.smithi112.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T00:51:59.654 INFO:teuthology.orchestra.run.smithi187.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T00:51:59.663 INFO:teuthology.orchestra.run.smithi187.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T00:51:59.693 INFO:teuthology.orchestra.run.smithi112.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T00:51:59.710 INFO:teuthology.orchestra.run.smithi187.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T00:51:59.718 INFO:teuthology.orchestra.run.smithi112.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T00:51:59.720 INFO:teuthology.orchestra.run.smithi112.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T00:51:59.727 INFO:teuthology.orchestra.run.smithi112.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T00:51:59.730 INFO:teuthology.orchestra.run.smithi112.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T00:51:59.735 INFO:teuthology.orchestra.run.smithi187.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T00:51:59.737 INFO:teuthology.orchestra.run.smithi187.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T00:51:59.743 INFO:teuthology.orchestra.run.smithi187.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T00:51:59.746 INFO:teuthology.orchestra.run.smithi187.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T00:51:59.857 INFO:teuthology.orchestra.run.smithi187.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T00:51:59.926 INFO:teuthology.orchestra.run.smithi112.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T00:51:59.944 INFO:teuthology.orchestra.run.smithi187.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T00:52:00.186 INFO:teuthology.orchestra.run.smithi112.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T00:52:00.197 INFO:teuthology.orchestra.run.smithi187.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T00:52:00.211 INFO:teuthology.orchestra.run.smithi112.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T00:52:00.424 INFO:teuthology.orchestra.run.smithi112.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T00:52:00.427 INFO:teuthology.orchestra.run.smithi112.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T00:52:00.432 INFO:teuthology.orchestra.run.smithi187.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T00:52:00.436 INFO:teuthology.orchestra.run.smithi187.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T00:52:00.446 INFO:teuthology.orchestra.run.smithi112.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T00:52:00.453 INFO:teuthology.orchestra.run.smithi112.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T00:52:00.454 INFO:teuthology.orchestra.run.smithi187.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T00:52:00.460 INFO:teuthology.orchestra.run.smithi187.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T00:52:00.486 INFO:teuthology.orchestra.run.smithi112.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T00:52:00.496 INFO:teuthology.orchestra.run.smithi187.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T00:52:00.499 INFO:teuthology.orchestra.run.smithi112.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T00:52:00.509 INFO:teuthology.orchestra.run.smithi187.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T00:52:00.591 INFO:teuthology.orchestra.run.smithi112.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T00:52:00.592 INFO:teuthology.orchestra.run.smithi112.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T00:52:00.593 INFO:teuthology.orchestra.run.smithi112.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T00:52:00.600 INFO:teuthology.orchestra.run.smithi187.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T00:52:00.602 INFO:teuthology.orchestra.run.smithi187.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T00:52:00.603 INFO:teuthology.orchestra.run.smithi187.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T00:52:00.709 INFO:teuthology.orchestra.run.smithi112.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T00:52:00.773 INFO:teuthology.orchestra.run.smithi112.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T00:52:00.782 INFO:teuthology.orchestra.run.smithi187.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T00:52:00.955 INFO:teuthology.orchestra.run.smithi112.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T00:52:00.956 INFO:teuthology.orchestra.run.smithi112.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T00:52:01.044 INFO:teuthology.orchestra.run.smithi187.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T00:52:01.046 INFO:teuthology.orchestra.run.smithi112.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T00:52:01.085 INFO:teuthology.orchestra.run.smithi112.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T00:52:01.121 INFO:teuthology.orchestra.run.smithi112.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T00:52:01.368 INFO:teuthology.orchestra.run.smithi112.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T00:52:01.771 INFO:teuthology.orchestra.run.smithi187.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T00:52:01.771 INFO:teuthology.orchestra.run.smithi187.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T00:52:01.788 INFO:teuthology.orchestra.run.smithi187.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T00:52:01.961 INFO:teuthology.orchestra.run.smithi187.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T00:52:02.033 INFO:teuthology.orchestra.run.smithi187.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T00:52:02.053 INFO:teuthology.orchestra.run.smithi187.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T00:52:02.229 INFO:teuthology.orchestra.run.smithi187.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T00:52:02.921 INFO:teuthology.orchestra.run.smithi187.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T00:52:02.922 INFO:teuthology.orchestra.run.smithi187.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T00:52:04.679 INFO:teuthology.orchestra.run.smithi112.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T00:52:04.699 INFO:teuthology.orchestra.run.smithi112.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T00:52:04.860 INFO:teuthology.orchestra.run.smithi112.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T00:52:06.628 INFO:teuthology.orchestra.run.smithi187.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T00:52:06.630 INFO:teuthology.orchestra.run.smithi187.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T00:52:06.635 INFO:teuthology.orchestra.run.smithi187.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T00:52:06.636 INFO:teuthology.orchestra.run.smithi187.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T00:52:06.637 INFO:teuthology.orchestra.run.smithi187.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T00:52:06.637 INFO:teuthology.orchestra.run.smithi187.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T00:52:07.431 INFO:teuthology.orchestra.run.smithi187.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T00:52:08.049 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 180 MB in 14s (13.3 MB/s) 2024-04-25T00:52:08.160 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T00:52:08.247 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T00:52:08.253 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T00:52:08.309 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T00:52:08.612 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T00:52:08.638 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T00:52:08.678 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:52:08.731 INFO:teuthology.orchestra.run.smithi112.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T00:52:08.951 INFO:teuthology.orchestra.run.smithi112.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T00:52:08.967 INFO:teuthology.orchestra.run.smithi112.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T00:52:08.979 INFO:teuthology.orchestra.run.smithi112.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T00:52:09.014 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T00:52:09.041 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T00:52:09.081 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T00:52:09.097 INFO:teuthology.orchestra.run.smithi112.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T00:52:09.409 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T00:52:09.435 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T00:52:09.506 INFO:teuthology.orchestra.run.smithi112.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T00:52:09.526 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:52:09.879 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T00:52:09.906 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T00:52:09.946 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:52:10.307 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T00:52:10.333 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T00:52:10.373 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:52:10.752 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T00:52:10.779 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:52:10.827 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:52:11.003 INFO:teuthology.orchestra.run.smithi112.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T00:52:11.003 INFO:teuthology.orchestra.run.smithi112.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T00:52:11.021 INFO:teuthology.orchestra.run.smithi112.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T00:52:11.064 INFO:teuthology.orchestra.run.smithi112.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T00:52:11.074 INFO:teuthology.orchestra.run.smithi112.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T00:52:11.076 INFO:teuthology.orchestra.run.smithi112.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T00:52:11.120 INFO:teuthology.orchestra.run.smithi112.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T00:52:11.230 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T00:52:11.256 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:52:11.297 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:52:11.404 INFO:teuthology.orchestra.run.smithi112.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T00:52:11.405 INFO:teuthology.orchestra.run.smithi112.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T00:52:11.633 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T00:52:11.659 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:52:11.699 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:52:12.353 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T00:52:12.380 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T00:52:12.420 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:52:12.739 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librados2. 2024-04-25T00:52:12.765 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:12.805 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:13.304 INFO:teuthology.orchestra.run.smithi112.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T00:52:13.307 INFO:teuthology.orchestra.run.smithi112.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T00:52:13.311 INFO:teuthology.orchestra.run.smithi112.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T00:52:13.312 INFO:teuthology.orchestra.run.smithi112.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T00:52:13.313 INFO:teuthology.orchestra.run.smithi112.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T00:52:13.314 INFO:teuthology.orchestra.run.smithi112.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T00:52:13.492 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librbd1. 2024-04-25T00:52:13.518 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:13.558 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:13.663 INFO:teuthology.orchestra.run.smithi112.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T00:52:14.013 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T00:52:14.040 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:14.080 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:14.268 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 180 MB in 20s (9,054 kB/s) 2024-04-25T00:52:14.374 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rados. 2024-04-25T00:52:14.401 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:14.410 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T00:52:14.441 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:14.504 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T00:52:14.509 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T00:52:14.550 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T00:52:14.752 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T00:52:14.768 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:14.795 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T00:52:14.802 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:14.810 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T00:52:14.853 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:52:15.021 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T00:52:15.037 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:15.079 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:15.173 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T00:52:15.199 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T00:52:15.239 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T00:52:15.399 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T00:52:15.426 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:15.466 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:15.593 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T00:52:15.619 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T00:52:15.701 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:52:15.794 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T00:52:15.820 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T00:52:15.860 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T00:52:16.038 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T00:52:16.064 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T00:52:16.096 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T00:52:16.104 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:52:16.123 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:16.155 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:16.382 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T00:52:16.408 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T00:52:16.441 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:52:16.491 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T00:52:16.517 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T00:52:16.558 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:52:16.869 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T00:52:16.884 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T00:52:16.927 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:52:17.094 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T00:52:17.121 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:52:17.170 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:52:17.213 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T00:52:17.229 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T00:52:17.263 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:52:17.548 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T00:52:17.558 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package lua5.1. 2024-04-25T00:52:17.574 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:52:17.585 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T00:52:17.614 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:52:17.625 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:52:17.909 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T00:52:17.935 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T00:52:17.936 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package lua-any. 2024-04-25T00:52:17.952 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T00:52:17.976 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:52:17.995 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T00:52:18.231 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package zip. 2024-04-25T00:52:18.246 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T00:52:18.292 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T00:52:18.320 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T00:52:18.346 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T00:52:18.387 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:52:18.684 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package unzip. 2024-04-25T00:52:18.710 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T00:52:18.723 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package librados2. 2024-04-25T00:52:18.742 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:52:18.749 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:18.790 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:19.121 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package luarocks. 2024-04-25T00:52:19.148 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T00:52:19.188 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:52:19.226 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package librbd1. 2024-04-25T00:52:19.252 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:19.293 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:19.579 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package librgw2. 2024-04-25T00:52:19.606 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:19.646 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:19.729 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T00:52:19.755 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:19.795 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:20.056 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-rados. 2024-04-25T00:52:20.071 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:20.114 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:20.215 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T00:52:20.230 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:20.265 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:20.434 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T00:52:20.461 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:20.509 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:20.535 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T00:52:20.561 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T00:52:20.601 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:52:20.745 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T00:52:20.760 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:20.803 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:20.921 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T00:52:20.947 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:20.987 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:21.106 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T00:52:21.132 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:21.164 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:21.391 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-common. 2024-04-25T00:52:21.417 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:21.457 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:21.467 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T00:52:21.494 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T00:52:21.534 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T00:52:21.843 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T00:52:21.870 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:21.962 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:22.314 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T00:52:22.341 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T00:52:22.381 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:52:22.650 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T00:52:22.666 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T00:52:22.708 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:52:22.738 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-base. 2024-04-25T00:52:22.765 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:22.814 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:23.003 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T00:52:23.030 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T00:52:23.070 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:52:23.359 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T00:52:23.365 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package lua5.1. 2024-04-25T00:52:23.385 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T00:52:23.391 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T00:52:23.431 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:52:23.476 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:52:23.760 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package lua-any. 2024-04-25T00:52:23.786 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T00:52:23.826 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T00:52:23.887 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T00:52:23.914 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T00:52:23.954 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:52:24.079 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package zip. 2024-04-25T00:52:24.094 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T00:52:24.128 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T00:52:24.249 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T00:52:24.276 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T00:52:24.315 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:52:24.382 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package unzip. 2024-04-25T00:52:24.409 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T00:52:24.440 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:52:24.635 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T00:52:24.661 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T00:52:24.701 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:52:24.855 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package luarocks. 2024-04-25T00:52:24.881 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T00:52:24.921 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:52:24.988 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T00:52:25.014 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T00:52:25.054 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:52:25.329 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package librgw2. 2024-04-25T00:52:25.356 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:25.357 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T00:52:25.384 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T00:52:25.395 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:25.424 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T00:52:25.727 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-portend. 2024-04-25T00:52:25.753 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T00:52:25.793 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T00:52:25.865 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T00:52:25.891 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:25.931 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:26.038 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T00:52:26.064 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T00:52:26.096 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T00:52:26.267 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T00:52:26.294 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T00:52:26.334 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:52:26.399 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T00:52:26.426 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T00:52:26.465 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:52:27.495 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T00:52:27.521 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T00:52:27.553 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T00:52:27.596 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T00:52:27.622 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:27.654 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:27.840 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T00:52:27.866 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T00:52:27.906 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T00:52:28.066 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-common. 2024-04-25T00:52:28.093 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:28.132 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:28.201 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T00:52:28.227 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T00:52:28.267 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:52:28.562 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-mako. 2024-04-25T00:52:28.588 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T00:52:28.628 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:52:28.974 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T00:52:29.000 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T00:52:29.040 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:52:29.310 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T00:52:29.336 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T00:52:29.368 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:52:29.513 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-base. 2024-04-25T00:52:29.539 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:29.589 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:29.654 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-webob. 2024-04-25T00:52:29.681 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T00:52:29.721 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:52:30.091 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T00:52:30.117 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T00:52:30.142 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T00:52:30.168 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T00:52:30.169 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:52:30.200 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:52:30.477 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T00:52:30.486 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T00:52:30.503 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T00:52:30.513 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T00:52:30.543 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:52:30.552 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:52:30.813 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-paste. 2024-04-25T00:52:30.840 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T00:52:30.864 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T00:52:30.871 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:52:30.891 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T00:52:30.931 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:52:31.191 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T00:52:31.207 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T00:52:31.234 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T00:52:31.249 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:52:31.260 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T00:52:31.300 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:52:31.494 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T00:52:31.517 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T00:52:31.552 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:52:31.561 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T00:52:31.588 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T00:52:31.620 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:52:31.839 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T00:52:31.865 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T00:52:31.905 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T00:52:31.914 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T00:52:31.941 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T00:52:31.981 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T00:52:32.217 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T00:52:32.242 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-portend. 2024-04-25T00:52:32.243 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T00:52:32.269 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T00:52:32.283 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:52:32.309 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T00:52:32.611 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T00:52:32.638 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T00:52:32.663 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T00:52:32.678 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T00:52:32.690 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T00:52:32.730 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T00:52:32.981 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T00:52:33.008 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T00:52:33.033 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-idna. 2024-04-25T00:52:33.047 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:52:33.059 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T00:52:33.099 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T00:52:33.369 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T00:52:33.396 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T00:52:33.418 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T00:52:33.445 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T00:52:33.452 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:52:33.484 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T00:52:33.779 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T00:52:33.806 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T00:52:33.846 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T00:52:33.881 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-requests. 2024-04-25T00:52:33.908 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T00:52:33.947 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:52:34.149 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T00:52:34.175 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T00:52:34.215 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:52:34.259 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T00:52:34.286 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T00:52:34.325 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:52:34.518 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-mako. 2024-04-25T00:52:34.545 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T00:52:34.585 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:52:34.595 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T00:52:34.622 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:34.661 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:34.880 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T00:52:34.907 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T00:52:34.946 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:52:35.034 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T00:52:35.060 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:35.092 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:35.216 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T00:52:35.243 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T00:52:35.282 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:52:35.386 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T00:52:35.413 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T00:52:35.453 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:52:35.585 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-webob. 2024-04-25T00:52:35.612 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T00:52:35.651 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:52:35.756 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T00:52:35.782 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T00:52:35.822 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T00:52:36.005 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T00:52:36.033 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T00:52:36.074 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:52:36.159 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T00:52:36.185 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:36.225 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:36.383 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T00:52:36.409 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T00:52:36.449 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:52:36.587 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T00:52:36.614 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:36.662 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:36.710 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-paste. 2024-04-25T00:52:36.737 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T00:52:36.768 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:52:37.097 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T00:52:37.112 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T00:52:37.155 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:52:37.266 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T00:52:37.293 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:37.332 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:37.667 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T00:52:37.693 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T00:52:37.724 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:52:38.002 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T00:52:38.029 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T00:52:38.069 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T00:52:38.364 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T00:52:38.369 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph. 2024-04-25T00:52:38.384 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:38.390 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T00:52:38.418 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:38.430 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:52:38.672 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T00:52:38.698 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:38.738 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:38.785 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T00:52:38.812 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T00:52:38.851 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T00:52:39.108 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T00:52:39.135 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:39.174 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:39.179 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-idna. 2024-04-25T00:52:39.206 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T00:52:39.246 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T00:52:39.544 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephadm. 2024-04-25T00:52:39.559 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:39.565 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T00:52:39.592 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T00:52:39.593 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:39.632 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:52:39.897 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T00:52:39.924 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T00:52:39.927 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-requests. 2024-04-25T00:52:39.953 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T00:52:39.963 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T00:52:39.993 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:52:40.291 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T00:52:40.305 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T00:52:40.318 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:52:40.331 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T00:52:40.349 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:52:40.371 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:52:40.641 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T00:52:40.668 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:40.707 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:41.112 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T00:52:41.139 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:41.179 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:41.288 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-babel. 2024-04-25T00:52:41.315 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:52:41.346 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:52:41.432 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T00:52:41.458 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T00:52:41.518 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:52:41.616 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T00:52:41.631 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T00:52:41.674 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:52:41.818 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T00:52:41.845 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T00:52:41.884 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T00:52:42.002 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T00:52:42.025 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:42.060 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:42.221 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T00:52:42.247 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:42.287 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:42.381 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T00:52:42.408 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T00:52:42.447 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T00:52:42.641 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T00:52:42.667 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:42.707 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:42.733 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-routes. 2024-04-25T00:52:42.760 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T00:52:42.791 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:52:43.137 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T00:52:43.163 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:43.203 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:43.277 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T00:52:43.304 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:43.344 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:44.372 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph. 2024-04-25T00:52:44.399 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:44.439 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:44.675 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T00:52:44.691 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T00:52:44.725 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:52:44.766 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T00:52:44.793 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:44.832 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:45.153 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T00:52:45.179 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:45.195 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T00:52:45.219 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:45.222 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T00:52:45.262 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:52:45.639 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package cephadm. 2024-04-25T00:52:45.641 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T00:52:45.666 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:45.667 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T00:52:45.705 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:45.793 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:52:46.016 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T00:52:46.043 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T00:52:46.083 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T00:52:46.087 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T00:52:46.114 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T00:52:46.146 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:52:46.385 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T00:52:46.401 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:52:46.443 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:52:46.864 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T00:52:46.892 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:46.921 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:47.449 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-babel. 2024-04-25T00:52:47.476 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T00:52:47.515 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:52:47.818 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T00:52:47.845 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T00:52:47.884 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:52:47.908 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T00:52:47.936 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T00:52:47.975 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T00:52:48.278 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T00:52:48.305 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T00:52:48.344 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:52:48.521 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T00:52:48.548 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:48.587 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:48.656 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T00:52:48.683 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T00:52:48.714 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:52:48.891 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T00:52:48.918 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T00:52:48.957 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T00:52:48.976 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T00:52:49.003 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T00:52:49.034 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T00:52:49.260 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-routes. 2024-04-25T00:52:49.287 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T00:52:49.326 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:52:49.355 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T00:52:49.382 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T00:52:49.421 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T00:52:49.655 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T00:52:49.682 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:49.738 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:49.766 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T00:52:49.794 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T00:52:49.833 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:52:50.161 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T00:52:50.189 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T00:52:50.227 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T00:52:50.533 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T00:52:50.561 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T00:52:50.599 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:52:50.910 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T00:52:50.925 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T00:52:50.960 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:52:51.296 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T00:52:51.312 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:51.355 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:51.463 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T00:52:51.490 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T00:52:51.530 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:52:51.683 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T00:52:51.710 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T00:52:51.749 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:52:51.883 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T00:52:51.910 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T00:52:52.027 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:52:52.035 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T00:52:52.052 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:52:52.085 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:52:52.321 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package jq. 2024-04-25T00:52:52.330 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T00:52:52.337 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:52:52.356 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T00:52:52.379 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T00:52:52.396 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:52:52.632 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package socat. 2024-04-25T00:52:52.648 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T00:52:52.690 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:52:53.002 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T00:52:53.029 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T00:52:53.056 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T00:52:53.068 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:52:53.083 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:53.138 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:53.439 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-test. 2024-04-25T00:52:53.466 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:53.505 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:54.117 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T00:52:54.144 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T00:52:54.183 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T00:52:54.486 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T00:52:54.513 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T00:52:54.552 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:52:54.873 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T00:52:54.900 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T00:52:54.939 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:52:55.251 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T00:52:55.278 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T00:52:55.317 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T00:52:55.587 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T00:52:55.603 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T00:52:55.637 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T00:52:55.793 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T00:52:55.821 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:55.859 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:55.949 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T00:52:55.976 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T00:52:56.015 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:52:56.239 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T00:52:56.266 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:56.305 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:56.327 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T00:52:56.354 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T00:52:56.393 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T00:52:56.608 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T00:52:56.636 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T00:52:56.674 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T00:52:56.705 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T00:52:56.733 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T00:52:56.771 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:52:56.927 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T00:52:56.943 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:56.985 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:57.271 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T00:52:57.298 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:57.344 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T00:52:57.346 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:57.359 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:52:57.393 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:57.632 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T00:52:57.659 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:57.707 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:57.713 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T00:52:57.740 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T00:52:57.780 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:52:58.043 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T00:52:58.071 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T00:52:58.109 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:52:58.141 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T00:52:58.168 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:52:58.208 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:52:58.430 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T00:52:58.458 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T00:52:58.468 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package jq. 2024-04-25T00:52:58.484 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T00:52:58.497 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:52:58.527 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T00:52:58.771 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package socat. 2024-04-25T00:52:58.787 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T00:52:58.821 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:52:58.926 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T00:52:58.954 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T00:52:59.001 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:52:59.116 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T00:52:59.132 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T00:52:59.174 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:52:59.481 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package pkg-config. 2024-04-25T00:52:59.508 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T00:52:59.512 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-test. 2024-04-25T00:52:59.539 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:52:59.556 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:52:59.578 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:52:59.834 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T00:52:59.861 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T00:52:59.900 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:53:00.329 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T00:53:00.356 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T00:53:00.394 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T00:53:00.697 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T00:53:00.725 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T00:53:00.764 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T00:53:01.092 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T00:53:01.120 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T00:53:01.159 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:53:01.470 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T00:53:01.497 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T00:53:01.536 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T00:53:01.783 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T00:53:01.810 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:53:01.849 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:01.873 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-py. 2024-04-25T00:53:01.900 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T00:53:01.947 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T00:53:02.251 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T00:53:02.253 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T00:53:02.278 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T00:53:02.280 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:53:02.317 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:53:02.319 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:02.564 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T00:53:02.592 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T00:53:02.639 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T00:53:02.763 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T00:53:02.791 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T00:53:02.829 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:53:02.900 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T00:53:02.927 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:53:02.966 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:03.099 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-toml. 2024-04-25T00:53:03.127 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T00:53:03.157 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T00:53:03.227 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T00:53:03.243 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T00:53:03.286 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:03.494 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T00:53:03.521 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T00:53:03.538 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T00:53:03.559 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:53:03.565 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:53:03.604 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:03.846 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T00:53:03.874 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T00:53:03.904 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:53:04.118 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T00:53:04.145 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T00:53:04.157 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T00:53:04.173 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T00:53:04.184 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:53:04.207 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:53:04.514 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T00:53:04.541 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T00:53:04.580 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:53:04.901 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T00:53:04.928 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T00:53:04.952 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package radosgw. 2024-04-25T00:53:04.967 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:53:04.980 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:53:05.019 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:05.372 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package pkg-config. 2024-04-25T00:53:05.400 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T00:53:05.430 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:53:05.667 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T00:53:05.683 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T00:53:05.725 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:53:05.821 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T00:53:05.838 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:53:05.880 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:06.137 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T00:53:06.165 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T00:53:06.194 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T00:53:06.203 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T00:53:06.452 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:53:06.506 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T00:53:06.533 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T00:53:06.552 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:53:06.572 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T00:53:06.797 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T00:53:06.901 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T00:53:06.928 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T00:53:06.969 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:53:07.116 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T00:53:07.278 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T00:53:07.306 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T00:53:07.345 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T00:53:07.521 INFO:teuthology.orchestra.run.smithi187.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T00:53:07.630 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:07.656 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-py. 2024-04-25T00:53:07.683 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T00:53:07.722 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T00:53:08.051 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T00:53:08.078 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T00:53:08.117 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:53:08.472 INFO:teuthology.orchestra.run.smithi187.stdout:Adding system user cephadm....done 2024-04-25T00:53:08.538 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T00:53:08.554 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T00:53:08.588 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:53:08.611 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:53:08.882 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-toml. 2024-04-25T00:53:08.888 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:53:08.910 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T00:53:08.949 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T00:53:09.139 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:53:09.260 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T00:53:09.273 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:53:09.288 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T00:53:09.326 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:53:09.533 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T00:53:09.655 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T00:53:09.683 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T00:53:09.721 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:53:09.792 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:53:09.893 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T00:53:09.999 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T00:53:10.027 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T00:53:10.066 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:53:10.362 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:53:10.722 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T00:53:10.837 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package radosgw. 2024-04-25T00:53:10.865 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:53:10.903 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:10.998 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T00:53:11.258 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T00:53:11.534 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:53:11.673 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T00:53:11.701 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T00:53:11.739 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:11.769 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:53:12.012 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:53:12.035 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T00:53:12.278 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T00:53:12.279 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:53:12.404 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T00:53:12.530 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:12.663 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T00:53:12.798 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:53:12.924 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:53:12.980 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T00:53:13.050 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T00:53:13.301 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:53:13.406 INFO:teuthology.orchestra.run.smithi112.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T00:53:13.427 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:53:13.480 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:13.803 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:53:14.217 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T00:53:14.250 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T00:53:14.315 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:53:14.373 INFO:teuthology.orchestra.run.smithi112.stdout:Adding system user cephadm....done 2024-04-25T00:53:14.441 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T00:53:14.529 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T00:53:14.800 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:53:14.806 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T00:53:14.909 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T00:53:15.040 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T00:53:15.141 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T00:53:15.160 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:53:15.392 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T00:53:15.411 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:53:15.652 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T00:53:15.730 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T00:53:15.778 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T00:53:15.855 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:53:16.088 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T00:53:16.448 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T00:53:16.557 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:53:16.708 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T00:53:16.825 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:53:16.926 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:53:16.967 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T00:53:17.027 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:53:17.252 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T00:53:17.303 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T00:53:17.520 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T00:53:17.696 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T00:53:17.779 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T00:53:17.947 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:53:18.089 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T00:53:18.323 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:53:18.331 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:18.608 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T00:53:18.608 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T00:53:18.734 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T00:53:18.860 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T00:53:18.901 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:53:19.136 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T00:53:19.227 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:53:19.262 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T00:53:19.478 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:53:19.604 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:19.721 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T00:53:19.905 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T00:53:20.137 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T00:53:20.169 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T00:53:20.233 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T00:53:20.256 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:53:20.359 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T00:53:20.719 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T00:53:20.845 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T00:53:21.112 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T00:53:21.362 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:53:21.380 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T00:53:21.622 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:53:21.681 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T00:53:21.757 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T00:53:21.816 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T00:53:22.016 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:53:22.134 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:53:22.234 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:53:22.503 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:53:22.518 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T00:53:22.753 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T00:53:22.777 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T00:53:22.903 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T00:53:23.005 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:53:23.029 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T00:53:23.139 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:53:23.306 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T00:53:23.441 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:53:23.681 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T00:53:23.692 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T00:53:23.924 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T00:53:23.926 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T00:53:24.160 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:53:24.326 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T00:53:24.411 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T00:53:24.619 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T00:53:24.670 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:53:24.796 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T00:53:24.920 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T00:53:25.106 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:53:25.232 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:53:25.263 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T00:53:25.514 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T00:53:25.525 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:53:25.615 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:25.802 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:53:25.908 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T00:53:26.044 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:53:26.170 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:53:26.226 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T00:53:26.296 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:53:26.386 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T00:53:26.631 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:53:26.782 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:53:27.175 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:53:27.399 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T00:53:27.426 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T00:53:27.527 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T00:53:27.651 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:53:27.785 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T00:53:27.786 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:53:27.912 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:53:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T00:53:28.170 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T00:53:28.230 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:53:28.296 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T00:53:28.540 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:53:28.589 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T00:53:28.816 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T00:53:28.832 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T00:53:29.018 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T00:53:29.083 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:53:29.210 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T00:53:29.278 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T00:53:29.519 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T00:53:29.546 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:53:29.770 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T00:53:29.806 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:53:30.038 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T00:53:30.183 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T00:53:30.289 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T00:53:30.476 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:53:30.532 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T00:53:30.791 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T00:53:30.794 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:53:30.921 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:30.926 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T00:53:31.047 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:31.182 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:53:31.211 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T00:53:31.337 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T00:53:31.492 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:31.594 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:31.638 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T00:53:31.711 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T00:53:31.922 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T00:53:31.988 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:32.114 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:32.165 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T00:53:32.240 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:32.274 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T00:53:32.401 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T00:53:32.454 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:53:32.454 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:53:32.490 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T00:53:32.736 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T00:53:32.862 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T00:53:32.958 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:33.059 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:33.168 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:33.255 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T00:53:33.286 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:33.437 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:53:33.506 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T00:53:33.615 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T00:53:33.866 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T00:53:34.001 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T00:53:34.319 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T00:53:34.620 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T00:53:34.665 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:34.791 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:34.892 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:34.904 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T00:53:34.993 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:35.102 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:35.116 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T00:53:35.228 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:35.375 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T00:53:35.652 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T00:53:35.780 INFO:teuthology.orchestra.run.smithi187.stdout:Adding group ceph....done 2024-04-25T00:53:35.911 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T00:53:36.173 INFO:teuthology.orchestra.run.smithi187.stdout:Adding system user ceph....done 2024-04-25T00:53:36.288 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T00:53:36.324 INFO:teuthology.orchestra.run.smithi187.stdout:Setting system user ceph properties....done 2024-04-25T00:53:36.337 INFO:teuthology.orchestra.run.smithi187.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T00:53:36.474 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T00:53:36.573 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T00:53:36.795 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T00:53:36.900 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T00:53:37.026 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:37.152 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:37.262 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:37.278 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T00:53:37.466 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:53:37.467 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:53:37.631 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:37.757 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:37.858 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T00:53:37.916 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:38.017 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:38.135 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:38.261 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:38.387 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:38.452 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:53:38.452 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:53:38.600 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:53:38.600 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T00:53:38.885 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:39.096 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:39.222 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:39.234 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T00:53:39.340 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:39.449 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:39.567 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T00:53:39.678 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:39.898 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:53:39.898 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:53:40.363 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:40.588 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:53:40.588 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:53:40.729 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:40.830 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:40.931 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:41.057 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:41.089 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:41.183 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:41.309 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:41.338 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:53:41.338 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:53:41.809 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:41.852 INFO:teuthology.orchestra.run.smithi112.stdout:Adding group ceph....done 2024-04-25T00:53:41.918 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:42.061 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:42.154 INFO:teuthology.orchestra.run.smithi112.stdout:Adding system user ceph....done 2024-04-25T00:53:42.272 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:53:42.273 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:53:42.323 INFO:teuthology.orchestra.run.smithi112.stdout:Setting system user ceph properties....done 2024-04-25T00:53:42.342 INFO:teuthology.orchestra.run.smithi112.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T00:53:42.482 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T00:53:42.753 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:42.826 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T00:53:42.904 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:43.031 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:43.181 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:43.319 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:43.529 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:53:43.529 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:53:43.534 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:53:43.627 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:53:43.771 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:53:44.021 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:44.138 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:44.571 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:53:44.571 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:53:44.905 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:53:45.249 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:45.550 INFO:teuthology.orchestra.run.smithi187.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:53:45.553 DEBUG:teuthology.orchestra.run.smithi187:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T00:53:45.612 INFO:teuthology.orchestra.run.smithi187.stdout:Reading package lists... 2024-04-25T00:53:45.614 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T00:53:45.813 INFO:teuthology.orchestra.run.smithi187.stdout:Building dependency tree... 2024-04-25T00:53:45.814 INFO:teuthology.orchestra.run.smithi187.stdout:Reading state information... 2024-04-25T00:53:46.001 INFO:teuthology.orchestra.run.smithi187.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T00:53:46.001 INFO:teuthology.orchestra.run.smithi187.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T00:53:46.001 INFO:teuthology.orchestra.run.smithi187.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T00:53:46.001 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:53:46.001 INFO:teuthology.orchestra.run.smithi187.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:53:46.001 INFO:teuthology.orchestra.run.smithi187.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:53:46.001 INFO:teuthology.orchestra.run.smithi187.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:53:46.001 INFO:teuthology.orchestra.run.smithi187.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:53:46.002 INFO:teuthology.orchestra.run.smithi187.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:53:46.002 INFO:teuthology.orchestra.run.smithi187.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:53:46.002 INFO:teuthology.orchestra.run.smithi187.stdout:The following additional packages will be installed: 2024-04-25T00:53:46.002 INFO:teuthology.orchestra.run.smithi187.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T00:53:46.002 INFO:teuthology.orchestra.run.smithi187.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T00:53:46.003 INFO:teuthology.orchestra.run.smithi187.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T00:53:46.003 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T00:53:46.003 INFO:teuthology.orchestra.run.smithi187.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:53:46.004 INFO:teuthology.orchestra.run.smithi187.stdout:Suggested packages: 2024-04-25T00:53:46.004 INFO:teuthology.orchestra.run.smithi187.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T00:53:46.004 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T00:53:46.004 INFO:teuthology.orchestra.run.smithi187.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T00:53:46.044 INFO:teuthology.orchestra.run.smithi187.stdout:The following NEW packages will be installed: 2024-04-25T00:53:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T00:53:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T00:53:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T00:53:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T00:53:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T00:53:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T00:53:46.045 INFO:teuthology.orchestra.run.smithi187.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:53:46.046 INFO:teuthology.orchestra.run.smithi187.stdout:The following packages will be upgraded: 2024-04-25T00:53:46.046 INFO:teuthology.orchestra.run.smithi187.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T00:53:46.126 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:46.162 INFO:teuthology.orchestra.run.smithi187.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T00:53:46.162 INFO:teuthology.orchestra.run.smithi187.stdout:Need to get 55.7 MB of archives. 2024-04-25T00:53:46.162 INFO:teuthology.orchestra.run.smithi187.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T00:53:46.162 INFO:teuthology.orchestra.run.smithi187.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T00:53:46.365 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:53:46.366 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T00:53:46.481 INFO:teuthology.orchestra.run.smithi187.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T00:53:46.493 INFO:teuthology.orchestra.run.smithi187.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T00:53:46.581 INFO:teuthology.orchestra.run.smithi187.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T00:53:46.582 INFO:teuthology.orchestra.run.smithi187.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T00:53:46.583 INFO:teuthology.orchestra.run.smithi187.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T00:53:46.611 INFO:teuthology.orchestra.run.smithi187.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T00:53:46.613 INFO:teuthology.orchestra.run.smithi187.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T00:53:46.620 INFO:teuthology.orchestra.run.smithi187.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T00:53:46.623 INFO:teuthology.orchestra.run.smithi187.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T00:53:46.624 INFO:teuthology.orchestra.run.smithi187.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T00:53:46.626 INFO:teuthology.orchestra.run.smithi187.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T00:53:46.630 INFO:teuthology.orchestra.run.smithi187.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T00:53:46.663 INFO:teuthology.orchestra.run.smithi187.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T00:53:46.664 INFO:teuthology.orchestra.run.smithi187.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T00:53:46.666 INFO:teuthology.orchestra.run.smithi187.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T00:53:46.827 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:47.040 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:53:47.040 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T00:53:47.396 INFO:teuthology.orchestra.run.smithi187.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T00:53:47.401 INFO:teuthology.orchestra.run.smithi187.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T00:53:47.401 INFO:teuthology.orchestra.run.smithi187.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T00:53:47.408 INFO:teuthology.orchestra.run.smithi187.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T00:53:47.408 INFO:teuthology.orchestra.run.smithi187.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T00:53:47.420 INFO:teuthology.orchestra.run.smithi187.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T00:53:47.424 INFO:teuthology.orchestra.run.smithi187.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T00:53:47.441 INFO:teuthology.orchestra.run.smithi187.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T00:53:47.441 INFO:teuthology.orchestra.run.smithi187.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T00:53:47.476 INFO:teuthology.orchestra.run.smithi187.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T00:53:47.476 INFO:teuthology.orchestra.run.smithi187.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T00:53:47.488 INFO:teuthology.orchestra.run.smithi187.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T00:53:47.495 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:47.521 INFO:teuthology.orchestra.run.smithi187.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T00:53:47.526 INFO:teuthology.orchestra.run.smithi187.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T00:53:47.528 INFO:teuthology.orchestra.run.smithi187.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T00:53:47.528 INFO:teuthology.orchestra.run.smithi187.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T00:53:47.529 INFO:teuthology.orchestra.run.smithi187.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T00:53:47.549 INFO:teuthology.orchestra.run.smithi187.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T00:53:47.551 INFO:teuthology.orchestra.run.smithi187.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T00:53:47.773 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:53:47.773 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T00:53:47.943 INFO:teuthology.orchestra.run.smithi187.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T00:53:47.943 INFO:teuthology.orchestra.run.smithi187.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T00:53:47.943 INFO:teuthology.orchestra.run.smithi187.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T00:53:47.943 INFO:teuthology.orchestra.run.smithi187.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T00:53:47.943 INFO:teuthology.orchestra.run.smithi187.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T00:53:47.944 INFO:teuthology.orchestra.run.smithi187.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T00:53:47.964 INFO:teuthology.orchestra.run.smithi187.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T00:53:47.964 INFO:teuthology.orchestra.run.smithi187.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T00:53:48.280 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:48.373 INFO:teuthology.orchestra.run.smithi187.stdout:Preconfiguring packages ... 2024-04-25T00:53:48.381 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:48.516 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:48.521 INFO:teuthology.orchestra.run.smithi187.stdout:Fetched 55.7 MB in 2s (29.2 MB/s) 2024-04-25T00:53:48.729 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:53:48.729 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T00:53:48.843 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:53:48.848 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:53:48.994 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:53:49.225 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:49.385 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:49.511 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:49.623 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T00:53:49.662 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T00:53:50.007 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:53:50.098 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:53:50.243 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:53:50.519 INFO:teuthology.orchestra.run.smithi187.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T00:53:50.806 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package flex. 2024-04-25T00:53:50.905 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:53:50.911 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T00:53:50.955 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T00:53:51.323 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:53:51.582 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:53:51.746 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T00:53:51.930 INFO:teuthology.orchestra.run.smithi112.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:53:51.933 DEBUG:teuthology.orchestra.run.smithi112:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T00:53:51.991 INFO:teuthology.orchestra.run.smithi112.stdout:Reading package lists... 2024-04-25T00:53:52.173 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:53:52.199 INFO:teuthology.orchestra.run.smithi112.stdout:Building dependency tree... 2024-04-25T00:53:52.200 INFO:teuthology.orchestra.run.smithi112.stdout:Reading state information... 2024-04-25T00:53:52.393 INFO:teuthology.orchestra.run.smithi112.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T00:53:52.393 INFO:teuthology.orchestra.run.smithi112.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T00:53:52.393 INFO:teuthology.orchestra.run.smithi112.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T00:53:52.393 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:53:52.393 INFO:teuthology.orchestra.run.smithi112.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:53:52.393 INFO:teuthology.orchestra.run.smithi112.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:53:52.394 INFO:teuthology.orchestra.run.smithi112.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:53:52.394 INFO:teuthology.orchestra.run.smithi112.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:53:52.394 INFO:teuthology.orchestra.run.smithi112.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:53:52.394 INFO:teuthology.orchestra.run.smithi112.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:53:52.395 INFO:teuthology.orchestra.run.smithi112.stdout:The following additional packages will be installed: 2024-04-25T00:53:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T00:53:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T00:53:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T00:53:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T00:53:52.395 INFO:teuthology.orchestra.run.smithi112.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:53:52.396 INFO:teuthology.orchestra.run.smithi112.stdout:Suggested packages: 2024-04-25T00:53:52.396 INFO:teuthology.orchestra.run.smithi112.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T00:53:52.396 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T00:53:52.396 INFO:teuthology.orchestra.run.smithi112.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T00:53:52.436 INFO:teuthology.orchestra.run.smithi112.stdout:The following NEW packages will be installed: 2024-04-25T00:53:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T00:53:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T00:53:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T00:53:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T00:53:52.438 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T00:53:52.441 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T00:53:52.441 INFO:teuthology.orchestra.run.smithi112.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T00:53:52.442 INFO:teuthology.orchestra.run.smithi112.stdout:The following packages will be upgraded: 2024-04-25T00:53:52.442 INFO:teuthology.orchestra.run.smithi112.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T00:53:52.499 INFO:teuthology.orchestra.run.smithi112.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T00:53:52.500 INFO:teuthology.orchestra.run.smithi112.stdout:Need to get 55.7 MB of archives. 2024-04-25T00:53:52.500 INFO:teuthology.orchestra.run.smithi112.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T00:53:52.500 INFO:teuthology.orchestra.run.smithi112.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T00:53:52.691 INFO:teuthology.orchestra.run.smithi112.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T00:53:52.700 INFO:teuthology.orchestra.run.smithi112.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T00:53:52.700 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:53:52.705 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:53:52.742 INFO:teuthology.orchestra.run.smithi112.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T00:53:52.744 INFO:teuthology.orchestra.run.smithi112.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T00:53:52.744 INFO:teuthology.orchestra.run.smithi112.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T00:53:52.756 INFO:teuthology.orchestra.run.smithi112.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T00:53:52.757 INFO:teuthology.orchestra.run.smithi112.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T00:53:52.759 INFO:teuthology.orchestra.run.smithi112.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T00:53:52.760 INFO:teuthology.orchestra.run.smithi112.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T00:53:52.761 INFO:teuthology.orchestra.run.smithi112.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T00:53:52.762 INFO:teuthology.orchestra.run.smithi112.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T00:53:52.769 INFO:teuthology.orchestra.run.smithi112.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T00:53:52.785 INFO:teuthology.orchestra.run.smithi112.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T00:53:52.785 INFO:teuthology.orchestra.run.smithi112.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T00:53:52.786 INFO:teuthology.orchestra.run.smithi112.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T00:53:52.829 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:53:53.212 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:53:53.245 INFO:teuthology.orchestra.run.smithi112.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T00:53:53.248 INFO:teuthology.orchestra.run.smithi112.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T00:53:53.248 INFO:teuthology.orchestra.run.smithi112.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T00:53:53.252 INFO:teuthology.orchestra.run.smithi112.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T00:53:53.253 INFO:teuthology.orchestra.run.smithi112.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T00:53:53.259 INFO:teuthology.orchestra.run.smithi112.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T00:53:53.261 INFO:teuthology.orchestra.run.smithi112.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T00:53:53.271 INFO:teuthology.orchestra.run.smithi112.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T00:53:53.272 INFO:teuthology.orchestra.run.smithi112.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T00:53:53.301 INFO:teuthology.orchestra.run.smithi112.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T00:53:53.301 INFO:teuthology.orchestra.run.smithi112.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T00:53:53.312 INFO:teuthology.orchestra.run.smithi112.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T00:53:53.332 INFO:teuthology.orchestra.run.smithi112.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T00:53:53.337 INFO:teuthology.orchestra.run.smithi112.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T00:53:53.337 INFO:teuthology.orchestra.run.smithi112.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T00:53:53.338 INFO:teuthology.orchestra.run.smithi112.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T00:53:53.338 INFO:teuthology.orchestra.run.smithi112.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T00:53:53.343 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:53:53.353 INFO:teuthology.orchestra.run.smithi112.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T00:53:53.356 INFO:teuthology.orchestra.run.smithi112.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T00:53:53.645 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:53:53.715 INFO:teuthology.orchestra.run.smithi112.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T00:53:53.715 INFO:teuthology.orchestra.run.smithi112.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T00:53:53.715 INFO:teuthology.orchestra.run.smithi112.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T00:53:53.716 INFO:teuthology.orchestra.run.smithi112.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T00:53:53.716 INFO:teuthology.orchestra.run.smithi112.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T00:53:53.716 INFO:teuthology.orchestra.run.smithi112.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T00:53:53.731 INFO:teuthology.orchestra.run.smithi112.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T00:53:53.732 INFO:teuthology.orchestra.run.smithi112.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T00:53:54.044 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package bison. 2024-04-25T00:53:54.086 INFO:teuthology.orchestra.run.smithi112.stdout:Preconfiguring packages ... 2024-04-25T00:53:54.148 INFO:teuthology.orchestra.run.smithi187.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:53:54.153 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T00:53:54.205 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:53:54.245 INFO:teuthology.orchestra.run.smithi112.stdout:Fetched 55.7 MB in 1s (43.3 MB/s) 2024-04-25T00:53:54.585 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:53:54.591 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:53:54.613 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T00:53:54.641 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T00:53:54.680 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:53:54.711 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:53:54.975 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package dump. 2024-04-25T00:53:55.002 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T00:53:55.032 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T00:53:55.286 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package indent. 2024-04-25T00:53:55.303 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:53:55.326 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T00:53:55.336 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:53:55.581 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T00:53:55.597 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T00:53:55.639 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:53:55.951 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T00:53:55.979 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T00:53:56.018 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:53:56.274 INFO:teuthology.orchestra.run.smithi112.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T00:53:56.287 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T00:53:56.303 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T00:53:56.381 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:53:56.518 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package flex. 2024-04-25T00:53:56.582 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T00:53:56.588 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T00:53:56.625 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T00:53:56.757 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T00:53:56.785 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T00:53:56.833 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:53:57.077 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T00:53:57.093 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T00:53:57.109 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:53:57.135 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:53:57.282 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T00:53:57.354 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T00:53:57.370 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T00:53:57.404 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T00:53:57.699 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T00:53:57.700 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:53:57.727 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T00:53:57.765 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:53:58.244 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:53:58.250 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:53:58.374 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:53:58.782 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T00:53:58.896 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T00:53:59.215 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:53:59.402 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T00:53:59.430 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:53:59.468 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:53:59.547 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package bison. 2024-04-25T00:53:59.650 INFO:teuthology.orchestra.run.smithi112.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T00:53:59.655 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T00:53:59.708 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:53:59.738 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T00:53:59.765 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:53:59.804 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:54:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T00:54:00.129 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T00:54:00.141 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T00:54:00.167 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:54:00.168 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:54:00.207 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:54:00.486 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package dump. 2024-04-25T00:54:00.510 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T00:54:00.514 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T00:54:00.538 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T00:54:00.553 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T00:54:00.568 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:54:00.832 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package indent. 2024-04-25T00:54:00.860 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:00.879 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T00:54:00.907 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:54:00.946 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:54:00.948 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:54:01.185 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T00:54:01.212 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T00:54:01.251 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:54:01.299 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T00:54:01.326 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:01.365 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:01.572 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T00:54:01.600 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T00:54:01.639 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:54:01.779 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T00:54:01.807 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:01.845 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:54:01.891 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T00:54:01.907 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T00:54:01.941 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:54:02.089 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T00:54:02.105 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:02.156 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:54:02.236 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T00:54:02.264 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T00:54:02.303 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:54:02.409 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T00:54:02.436 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:54:02.475 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:54:02.556 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T00:54:02.583 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T00:54:02.622 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:54:02.875 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T00:54:02.891 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T00:54:02.933 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T00:54:02.970 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T00:54:02.998 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T00:54:03.037 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:54:03.220 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T00:54:03.236 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T00:54:03.270 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:54:03.590 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package modemmanager. 2024-04-25T00:54:03.619 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:54:03.657 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:54:04.314 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package network-manager. 2024-04-25T00:54:04.341 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:54:04.389 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:54:04.923 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T00:54:04.950 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:54:04.990 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:54:05.013 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package ppp. 2024-04-25T00:54:05.041 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T00:54:05.251 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T00:54:05.267 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T00:54:05.300 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:54:05.309 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:54:05.618 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T00:54:05.620 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T00:54:05.645 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T00:54:05.647 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:54:05.684 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T00:54:05.686 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:54:05.912 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T00:54:05.928 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T00:54:05.962 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T00:54:06.040 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T00:54:06.067 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T00:54:06.106 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:54:06.248 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T00:54:06.276 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T00:54:06.314 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T00:54:06.351 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T00:54:06.378 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:54:06.409 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:54:06.745 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T00:54:06.745 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T00:54:06.773 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:06.773 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:06.811 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:06.812 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:07.149 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T00:54:07.176 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:07.215 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:54:07.407 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T00:54:07.436 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T00:54:07.484 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T00:54:07.493 INFO:teuthology.orchestra.run.smithi187.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T00:54:07.501 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:07.522 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-common (238) ... 2024-04-25T00:54:07.559 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:54:07.804 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T00:54:07.831 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T00:54:07.870 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:54:07.973 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T00:54:08.001 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:08.042 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:08.449 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T00:54:08.477 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T00:54:08.516 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:54:08.871 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package modemmanager. 2024-04-25T00:54:08.896 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T00:54:08.937 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:54:09.192 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql. 2024-04-25T00:54:09.209 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T00:54:09.250 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql (14+238) ... 2024-04-25T00:54:09.494 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T00:54:09.510 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T00:54:09.517 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package network-manager. 2024-04-25T00:54:09.545 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T00:54:09.552 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T00:54:09.592 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:54:09.821 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T00:54:09.849 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T00:54:09.896 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T00:54:10.150 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package ppp. 2024-04-25T00:54:10.177 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T00:54:10.307 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T00:54:10.335 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T00:54:10.374 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:54:10.444 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:54:10.652 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package tcl. 2024-04-25T00:54:10.680 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T00:54:10.723 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T00:54:10.729 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T00:54:10.745 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T00:54:10.779 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T00:54:11.024 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T00:54:11.040 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T00:54:11.071 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T00:54:11.082 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T00:54:11.099 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T00:54:11.137 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:54:11.385 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T00:54:11.413 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T00:54:11.451 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T00:54:11.458 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T00:54:11.475 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T00:54:11.508 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T00:54:11.781 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T00:54:11.808 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:11.847 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:12.295 INFO:teuthology.orchestra.run.smithi187.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T00:54:12.323 INFO:teuthology.orchestra.run.smithi187.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T00:54:12.370 INFO:teuthology.orchestra.run.smithi187.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:54:12.435 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T00:54:12.463 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T00:54:12.521 INFO:teuthology.orchestra.run.smithi112.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T00:54:12.550 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking postgresql-common (238) ... 2024-04-25T00:54:12.682 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T00:54:12.901 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T00:54:13.001 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T00:54:13.027 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:54:13.029 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T00:54:13.070 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:13.153 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:13.295 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:54:13.413 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:54:13.514 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:13.983 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T00:54:14.061 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:54:14.404 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package postgresql. 2024-04-25T00:54:14.421 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T00:54:14.421 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:54:14.421 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:54:14.462 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking postgresql (14+238) ... 2024-04-25T00:54:14.698 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T00:54:14.726 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T00:54:14.756 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T00:54:14.906 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:54:14.967 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T00:54:14.983 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T00:54:15.015 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:54:15.024 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T00:54:15.141 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:54:15.267 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T00:54:15.344 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T00:54:15.372 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T00:54:15.393 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:54:15.410 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:54:15.519 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:54:15.645 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:54:15.688 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package tcl. 2024-04-25T00:54:15.716 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T00:54:15.788 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:54:15.794 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T00:54:16.100 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T00:54:16.128 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T00:54:16.158 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:54:16.376 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:54:16.502 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T00:54:16.504 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T00:54:16.532 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T00:54:16.570 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T00:54:16.612 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T00:54:16.679 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:54:16.805 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:54:16.914 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T00:54:16.966 INFO:teuthology.orchestra.run.smithi112.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T00:54:16.994 INFO:teuthology.orchestra.run.smithi112.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T00:54:17.015 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T00:54:17.041 INFO:teuthology.orchestra.run.smithi112.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:54:17.116 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:54:17.210 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T00:54:17.276 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:54:17.320 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T00:54:17.521 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T00:54:17.647 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T00:54:17.773 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:17.777 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:54:17.900 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T00:54:17.954 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:54:18.025 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T00:54:18.105 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:54:18.151 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:18.206 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T00:54:18.357 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T00:54:18.483 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:54:18.671 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T00:54:18.757 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T00:54:18.955 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:18.955 INFO:teuthology.orchestra.run.smithi187.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T00:54:18.955 INFO:teuthology.orchestra.run.smithi187.stdout:which means they are currently configured by ifupdown: 2024-04-25T00:54:18.955 INFO:teuthology.orchestra.run.smithi187.stdout:- enp3s0f1 2024-04-25T00:54:18.955 INFO:teuthology.orchestra.run.smithi187.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T00:54:18.955 INFO:teuthology.orchestra.run.smithi187.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T00:54:18.956 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:19.077 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T00:54:19.136 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:54:19.136 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T00:54:19.354 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T00:54:19.635 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T00:54:19.675 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T00:54:19.753 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T00:54:19.887 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T00:54:20.013 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T00:54:20.148 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T00:54:20.154 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:54:20.264 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:54:20.273 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T00:54:20.382 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T00:54:20.399 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T00:54:20.508 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-common (238) ... 2024-04-25T00:54:20.525 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T00:54:21.352 INFO:teuthology.orchestra.run.smithi187.stdout:Adding user postgres to group ssl-cert 2024-04-25T00:54:21.394 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T00:54:21.571 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:21.571 INFO:teuthology.orchestra.run.smithi187.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T00:54:21.780 INFO:teuthology.orchestra.run.smithi187.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T00:54:21.780 INFO:teuthology.orchestra.run.smithi187.stdout:Removing obsolete dictionary files: 2024-04-25T00:54:21.807 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T00:54:21.901 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T00:54:21.967 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:54:22.093 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T00:54:22.219 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T00:54:22.339 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T00:54:22.345 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T00:54:22.446 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T00:54:22.536 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T00:54:22.606 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T00:54:22.731 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:22.732 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T00:54:22.908 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T00:54:23.034 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:54:23.065 INFO:teuthology.orchestra.run.smithi187.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T00:54:23.086 INFO:teuthology.orchestra.run.smithi187.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T00:54:23.140 INFO:teuthology.orchestra.run.smithi187.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout:This user must also own the server process. 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout:The default text search configuration will be set to "english". 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout:Data page checksums are disabled. 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout:creating subdirectories ... ok 2024-04-25T00:54:23.141 INFO:teuthology.orchestra.run.smithi187.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T00:54:23.157 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default max_connections ... 100 2024-04-25T00:54:23.160 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T00:54:23.178 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default shared_buffers ... 128MB 2024-04-25T00:54:23.187 INFO:teuthology.orchestra.run.smithi187.stdout:selecting default time zone ... Etc/UTC 2024-04-25T00:54:23.188 INFO:teuthology.orchestra.run.smithi187.stdout:creating configuration files ... ok 2024-04-25T00:54:23.304 INFO:teuthology.orchestra.run.smithi187.stdout:running bootstrap script ... ok 2024-04-25T00:54:23.328 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T00:54:23.438 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T00:54:23.924 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:23.924 INFO:teuthology.orchestra.run.smithi112.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T00:54:23.924 INFO:teuthology.orchestra.run.smithi112.stdout:which means they are currently configured by ifupdown: 2024-04-25T00:54:23.924 INFO:teuthology.orchestra.run.smithi112.stdout:- enp3s0f1 2024-04-25T00:54:23.924 INFO:teuthology.orchestra.run.smithi112.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T00:54:23.924 INFO:teuthology.orchestra.run.smithi112.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T00:54:23.924 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:23.961 INFO:teuthology.orchestra.run.smithi187.stdout:performing post-bootstrap initialization ... ok 2024-04-25T00:54:24.056 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T00:54:24.386 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T00:54:24.596 INFO:teuthology.orchestra.run.smithi187.stdout:syncing data to disk ... ok 2024-04-25T00:54:24.724 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T00:54:24.964 INFO:teuthology.orchestra.run.smithi187.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T00:54:25.249 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T00:54:25.392 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T00:54:25.510 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T00:54:25.619 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up postgresql-common (238) ... 2024-04-25T00:54:26.814 INFO:teuthology.orchestra.run.smithi112.stdout:Adding user postgres to group ssl-cert 2024-04-25T00:54:27.025 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:27.025 INFO:teuthology.orchestra.run.smithi112.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T00:54:27.243 INFO:teuthology.orchestra.run.smithi112.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T00:54:27.243 INFO:teuthology.orchestra.run.smithi112.stdout:Removing obsolete dictionary files: 2024-04-25T00:54:27.708 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:54:27.811 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T00:54:27.834 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T00:54:27.960 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up postgresql (14+238) ... 2024-04-25T00:54:28.087 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:54:28.154 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T00:54:28.196 INFO:teuthology.orchestra.run.smithi187.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:54:28.425 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:54:28.425 INFO:teuthology.orchestra.run.smithi187.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:54:28.475 INFO:teuthology.orchestra.run.smithi112.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T00:54:28.494 INFO:teuthology.orchestra.run.smithi112.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout:This user must also own the server process. 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout:The default text search configuration will be set to "english". 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout:Data page checksums are disabled. 2024-04-25T00:54:28.536 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:28.537 INFO:teuthology.orchestra.run.smithi112.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T00:54:28.537 INFO:teuthology.orchestra.run.smithi112.stdout:creating subdirectories ... ok 2024-04-25T00:54:28.537 INFO:teuthology.orchestra.run.smithi112.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T00:54:28.552 INFO:teuthology.orchestra.run.smithi112.stdout:selecting default max_connections ... 100 2024-04-25T00:54:28.575 INFO:teuthology.orchestra.run.smithi112.stdout:selecting default shared_buffers ... 128MB 2024-04-25T00:54:28.584 INFO:teuthology.orchestra.run.smithi112.stdout:selecting default time zone ... Etc/UTC 2024-04-25T00:54:28.585 INFO:teuthology.orchestra.run.smithi112.stdout:creating configuration files ... ok 2024-04-25T00:54:28.699 INFO:teuthology.orchestra.run.smithi112.stdout:running bootstrap script ... ok 2024-04-25T00:54:28.948 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:54:29.024 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T00:54:29.327 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:54:29.374 INFO:teuthology.orchestra.run.smithi112.stdout:performing post-bootstrap initialization ... ok 2024-04-25T00:54:29.496 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:54:30.025 INFO:teuthology.orchestra.run.smithi112.stdout:syncing data to disk ... ok 2024-04-25T00:54:30.419 INFO:teuthology.orchestra.run.smithi112.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T00:54:30.824 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T00:54:30.915 INFO:teuthology.orchestra.run.smithi187.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:54:31.434 INFO:teuthology.orchestra.run.smithi187.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:54:31.440 DEBUG:teuthology.parallel:result is None 2024-04-25T00:54:33.169 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:54:33.287 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T00:54:33.388 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up postgresql (14+238) ... 2024-04-25T00:54:33.505 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T00:54:33.631 INFO:teuthology.orchestra.run.smithi112.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T00:54:33.894 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:54:33.894 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T00:54:34.493 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T00:54:34.593 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T00:54:34.887 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T00:54:35.040 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T00:54:36.711 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T00:54:36.837 INFO:teuthology.orchestra.run.smithi112.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T00:54:37.334 INFO:teuthology.orchestra.run.smithi112.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T00:54:37.339 DEBUG:teuthology.parallel:result is None 2024-04-25T00:54:37.339 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:54:37.696 DEBUG:teuthology.orchestra.run.smithi112:> dpkg-query -W -f '${Version}' ceph 2024-04-25T00:54:37.715 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:54:37.716 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:54:37.716 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T00:54:37.718 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:54:37.862 DEBUG:teuthology.orchestra.run.smithi187:> dpkg-query -W -f '${Version}' ceph 2024-04-25T00:54:37.882 INFO:teuthology.orchestra.run.smithi187.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:54:37.882 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:54:37.882 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T00:54:37.887 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T00:54:37.887 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:37.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:54:37.901 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:54:37.902 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:54:37.938 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T00:54:37.938 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:37.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:54:37.957 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:54:38.013 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:54:38.013 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:54:38.026 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:54:38.080 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T00:54:38.080 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:38.080 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:54:38.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:54:38.148 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:54:38.149 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:54:38.161 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:54:38.216 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T00:54:38.216 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:38.217 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:54:38.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:54:38.284 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:54:38.284 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:54:38.297 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:54:38.352 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T00:54:38.444 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T00:54:38.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2024-04-25T00:54:38.447 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2024-04-25T00:54:38.459 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T00:54:38.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T00:54:38.503 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T00:54:38.516 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T00:54:38.516 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T00:54:38.558 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T00:54:38.576 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T00:54:38.576 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T00:54:38.576 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T00:54:38.577 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:38.577 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T00:54:38.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T00:54:38.692 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:54:38.692 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T00:54:38.697 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T00:54:38.777 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T00:54:38.778 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T00:54:38.778 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285', 'branch': 'squid', 'description': 'fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672285', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4w3XTBD3z3iI/FJ0TDJz24aQSOg0wgpZEJWbM2zwmoU5cppJBwtxtTf2VrZri4GLiwfDKYSxwe32Eb7UzdZk8=', 'smithi187.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNMI9STWpOkq+7tF/jBvaEKwDNQ9dzEZTLukGTCCNQDFKQkri1l5VVzrOTwYMlImxhfXnlgrD3G/FfhHvX3p0Kg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cephfs_shell']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T00:54:38.778 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T00:54:38.781 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T00:54:38.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T00:54:38.827 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T00:54:38.881 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:38.882 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:54:38.887 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:54:38.887 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2024-04-25T00:54:38.933 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:54:38.933 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:54:38.933 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 890 Links: 1 2024-04-25T00:54:38.934 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:54:38.934 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 00:48:27.251140250 +0000 2024-04-25T00:54:38.934 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 00:48:26.991142572 +0000 2024-04-25T00:54:38.934 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 00:48:26.991142572 +0000 2024-04-25T00:54:38.934 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2024-04-25T00:54:38.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:54:38.987 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T00:54:38.987 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T00:54:38.988 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000415901 s, 1.2 MB/s 2024-04-25T00:54:38.989 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:54:39.038 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2024-04-25T00:54:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:54:39.085 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:54:39.085 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-04-25T00:54:39.085 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:54:39.086 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 00:48:27.503137999 +0000 2024-04-25T00:54:39.086 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 00:48:27.499138035 +0000 2024-04-25T00:54:39.086 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 00:48:27.499138035 +0000 2024-04-25T00:54:39.086 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2024-04-25T00:54:39.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:54:39.139 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T00:54:39.140 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T00:54:39.140 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.00043818 s, 1.2 MB/s 2024-04-25T00:54:39.141 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:54:39.190 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2024-04-25T00:54:39.237 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:54:39.237 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:54:39.237 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-04-25T00:54:39.237 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:54:39.237 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 00:48:28.263131210 +0000 2024-04-25T00:54:39.237 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 00:48:28.011133462 +0000 2024-04-25T00:54:39.237 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 00:48:28.011133462 +0000 2024-04-25T00:54:39.238 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2024-04-25T00:54:39.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:54:39.291 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T00:54:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T00:54:39.292 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000393044 s, 1.3 MB/s 2024-04-25T00:54:39.293 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:54:39.343 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2024-04-25T00:54:39.389 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:54:39.389 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:54:39.390 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 944 Links: 1 2024-04-25T00:54:39.390 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:54:39.390 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2024-04-25 00:48:28.759126780 +0000 2024-04-25T00:54:39.390 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2024-04-25 00:48:28.519128924 +0000 2024-04-25T00:54:39.390 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2024-04-25 00:48:28.519128924 +0000 2024-04-25T00:54:39.390 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: - 2024-04-25T00:54:39.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:54:39.444 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2024-04-25T00:54:39.444 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2024-04-25T00:54:39.444 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000361102 s, 1.4 MB/s 2024-04-25T00:54:39.445 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:54:39.496 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:39.496 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:54:39.496 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:54:39.502 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:54:39.502 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2024-04-25T00:54:39.549 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:54:39.549 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:54:39.549 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-04-25T00:54:39.549 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:54:39.550 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 00:48:27.255124402 +0000 2024-04-25T00:54:39.550 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 00:48:26.983126832 +0000 2024-04-25T00:54:39.550 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 00:48:26.983126832 +0000 2024-04-25T00:54:39.550 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T00:54:39.550 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:54:39.604 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T00:54:39.604 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T00:54:39.604 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000453013 s, 1.1 MB/s 2024-04-25T00:54:39.605 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:54:39.655 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2024-04-25T00:54:39.701 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:54:39.701 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:54:39.701 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 903 Links: 1 2024-04-25T00:54:39.701 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:54:39.701 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 00:48:27.755119936 +0000 2024-04-25T00:54:39.701 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 00:48:27.491122294 +0000 2024-04-25T00:54:39.701 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 00:48:27.491122294 +0000 2024-04-25T00:54:39.701 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T00:54:39.701 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:54:39.756 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T00:54:39.756 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T00:54:39.756 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000393302 s, 1.3 MB/s 2024-04-25T00:54:39.757 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:54:39.806 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2024-04-25T00:54:39.853 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:54:39.853 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:54:39.853 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 916 Links: 1 2024-04-25T00:54:39.853 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:54:39.853 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 00:48:28.311114969 +0000 2024-04-25T00:54:39.853 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 00:48:28.031117471 +0000 2024-04-25T00:54:39.853 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 00:48:28.031117471 +0000 2024-04-25T00:54:39.853 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T00:54:39.854 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:54:39.907 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T00:54:39.907 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T00:54:39.907 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000402003 s, 1.3 MB/s 2024-04-25T00:54:39.908 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:54:39.958 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2024-04-25T00:54:40.009 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:54:40.009 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:54:40.009 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-04-25T00:54:40.009 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:54:40.009 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2024-04-25 00:48:28.827110361 +0000 2024-04-25T00:54:40.009 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2024-04-25 00:48:28.559112754 +0000 2024-04-25T00:54:40.009 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2024-04-25 00:48:28.559112754 +0000 2024-04-25T00:54:40.010 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2024-04-25T00:54:40.010 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:54:40.063 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2024-04-25T00:54:40.063 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2024-04-25T00:54:40.063 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000388703 s, 1.3 MB/s 2024-04-25T00:54:40.064 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:54:40.114 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:40.114 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T00:54:40.114 INFO:tasks.ceph:Generating config... 2024-04-25T00:54:40.115 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T00:54:40.115 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T00:54:40.115 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T00:54:40.116 INFO:tasks.ceph:[client] pid file = 2024-04-25T00:54:40.116 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T00:54:40.116 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T00:54:40.116 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T00:54:40.116 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T00:54:40.116 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T00:54:40.116 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T00:54:40.116 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T00:54:40.117 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T00:54:40.118 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T00:54:40.118 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T00:54:40.118 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T00:54:40.118 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T00:54:40.118 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T00:54:40.118 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T00:54:40.118 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T00:54:40.118 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T00:54:40.118 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T00:54:40.119 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T00:54:40.120 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T00:54:40.120 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T00:54:40.120 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T00:54:40.155 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T00:54:40.171 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T00:54:40.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:54:40.306 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.112'), ('mon.b', '172.21.15.187'), ('mon.c', '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]')] 2024-04-25T00:54:40.306 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T00:54:40.307 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:40.307 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T00:54:40.354 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.112 --add b 172.21.15.187 --addv c '[v2:172.21.15.187:3301,v1:172.21.15.187:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: generated fsid 86f864a3-be50-4513-98ca-86783428a0f8 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:setting min_mon_release = quincy 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:epoch 0 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:fsid 86f864a3-be50-4513-98ca-86783428a0f8 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:last_changed 2024-04-25T00:54:40.443329+0000 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:created 2024-04-25T00:54:40.443329+0000 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:min_mon_release 17 (quincy) 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:election_strategy: 1 2024-04-25T00:54:40.440 INFO:teuthology.orchestra.run.smithi112.stdout:0: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.a 2024-04-25T00:54:40.441 INFO:teuthology.orchestra.run.smithi112.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2024-04-25T00:54:40.441 INFO:teuthology.orchestra.run.smithi112.stdout:2: [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] mon.c 2024-04-25T00:54:40.441 INFO:teuthology.orchestra.run.smithi112.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T00:54:40.442 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T00:54:40.489 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 86f864a3-be50-4513-98ca-86783428a0f8... 2024-04-25T00:54:40.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:54:40.534 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2024-04-25T00:54:40.563 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2024-04-25T00:54:40.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2024-04-25T00:54:40.565 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = 86f864a3-be50-4513-98ca-86783428a0f8 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:54:40.566 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2024-04-25T00:54:40.567 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: bdev async discard = True 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: bdev enable discard = True 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore allocator = bitmap 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 1/20 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 1/20 2024-04-25T00:54:40.568 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 4/10 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T00:54:40.569 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2024-04-25T00:54:40.570 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: mon down mkfs grace = 300 2024-04-25T00:54:40.571 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2024-04-25T00:54:40.572 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = True 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = True 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2024-04-25T00:54:40.573 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2024-04-25T00:54:40.574 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2024-04-25T00:54:40.574 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2024-04-25T00:54:40.574 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T00:54:40.574 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2024-04-25T00:54:40.574 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T00:54:40.575 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.576 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.112,172.21.15.187,[v2:172.21.15.187:3301,v1:172.21.15.187:6790]" 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 86f864a3-be50-4513-98ca-86783428a0f8 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2024-04-25T00:54:40.577 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock profile = high_recovery_ops 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug trim objects = true 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2024-04-25T00:54:40.578 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: bdev async discard = True 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: bdev enable discard = True 2024-04-25T00:54:40.579 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore allocator = bitmap 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 1/20 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 1/20 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 4/10 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2024-04-25T00:54:40.580 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2024-04-25T00:54:40.581 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.582 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: mon down mkfs grace = 300 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: mon op complaint time = 120 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: client mount timeout = 600 2024-04-25T00:54:40.583 INFO:teuthology.orchestra.run.smithi187.stdout: debug client = 20 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = "" 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout:[mds] 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds = 20 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: debug mds balancer = 20 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug frag = True 2024-04-25T00:54:40.584 INFO:teuthology.orchestra.run.smithi187.stdout: mds debug scatterstat = True 2024-04-25T00:54:40.585 INFO:teuthology.orchestra.run.smithi187.stdout: mds op complaint time = 180 2024-04-25T00:54:40.585 INFO:teuthology.orchestra.run.smithi187.stdout: mds verify scatter = True 2024-04-25T00:54:40.585 INFO:teuthology.orchestra.run.smithi187.stdout: osd op complaint time = 180 2024-04-25T00:54:40.585 INFO:teuthology.orchestra.run.smithi187.stdout: rados mon op timeout = 900 2024-04-25T00:54:40.585 INFO:teuthology.orchestra.run.smithi187.stdout: rados osd op timeout = 900 2024-04-25T00:54:40.586 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T00:54:40.586 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T00:54:40.678 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T00:54:40.678 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:40.678 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T00:54:40.683 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:40.683 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T00:54:40.730 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:54:40.730 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:40.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T00:54:40.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:54:40.795 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:54:40.795 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:40.842 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:54:40.842 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:54:40.842 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T00:54:40.842 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T00:54:40.866 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:54:40.866 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:40.913 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T00:54:40.913 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T00:54:40.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T00:54:40.958 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T00:54:40.959 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T00:54:41.001 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T00:54:41.013 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T00:54:41.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T00:54:41.055 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T00:54:41.057 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T00:54:41.112 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T00:54:41.157 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T00:54:41.161 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T00:54:41.173 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T00:54:41.173 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:54:41.207 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T00:54:41.218 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T00:54:41.218 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T00:54:41.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T00:54:41.272 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:41.272 INFO:tasks.ceph:role: osd.0 2024-04-25T00:54:41.272 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:54:41.273 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T00:54:41.336 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:54:41.336 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:54:41.336 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:54:41.336 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:54:41.336 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:54:41.336 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:54:41.337 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:54:41.337 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:54:41.337 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:54:41.337 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:54:41.826 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T00:54:41.828 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T00:54:41.828 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T00:54:41.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T00:54:41.991 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:54:41.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:41.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T00:54:42.048 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:42.048 INFO:tasks.ceph:role: osd.1 2024-04-25T00:54:42.048 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:54:42.048 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:54:42.113 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:54:42.114 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:54:42.562 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T00:54:42.564 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T00:54:42.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T00:54:42.655 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T00:54:42.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:42.667 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:54:42.667 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T00:54:42.720 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:42.720 INFO:tasks.ceph:role: osd.2 2024-04-25T00:54:42.720 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:54:42.720 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:54:42.781 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:54:42.782 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:54:43.245 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T00:54:43.247 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T00:54:43.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T00:54:43.373 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T00:54:43.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:43.384 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:54:43.384 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T00:54:43.441 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:43.441 INFO:tasks.ceph:role: osd.3 2024-04-25T00:54:43.441 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:54:43.441 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T00:54:43.505 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:54:43.505 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:54:43.505 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:54:43.505 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:54:43.505 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:54:43.506 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:54:43.506 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:54:43.506 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:54:43.506 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:54:43.506 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:54:44.008 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2024-04-25T00:54:44.010 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2024-04-25T00:54:44.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T00:54:44.162 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T00:54:44.172 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:54:44.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:44.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:44.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:44.279+0000 7f6be6a2e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T00:54:44.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:44.283+0000 7f6be6a2e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T00:54:44.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:44.283+0000 7f6be6a2e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:54:44.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:44.283+0000 7f6be6a2e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:54:44.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:44.283+0000 7f6be6a2e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T00:54:44.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:44.291+0000 7f6be6a2e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T00:54:47.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T00:54:47.276 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:47.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:47.359+0000 7ff6690657c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T00:54:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:47.359+0000 7ff6690657c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T00:54:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:47.359+0000 7ff6690657c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:54:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:47.359+0000 7ff6690657c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:54:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:47.359+0000 7ff6690657c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T00:54:47.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:47.359+0000 7ff6690657c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T00:54:50.356 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T00:54:50.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:50.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:50.499+0000 7f50d0dd47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T00:54:50.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:50.503+0000 7f50d0dd47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T00:54:50.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:50.503+0000 7f50d0dd47c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:54:50.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:50.503+0000 7f50d0dd47c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:54:50.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:50.503+0000 7f50d0dd47c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T00:54:50.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:50.503+0000 7f50d0dd47c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T00:54:53.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T00:54:53.533 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:53.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:53.615+0000 7f73fb06f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T00:54:53.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:53.615+0000 7f73fb06f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T00:54:53.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:53.615+0000 7f73fb06f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:54:53.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:53.615+0000 7f73fb06f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:54:53.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:53.615+0000 7f73fb06f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T00:54:53.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:54:53.615+0000 7f73fb06f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T00:54:56.591 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T00:54:56.646 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T00:54:56.661 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:56.661 INFO:tasks.ceph:role: osd.4 2024-04-25T00:54:56.661 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:54:56.661 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:54:56.724 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:54:56.725 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:54:57.178 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T00:54:57.181 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T00:54:57.181 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T00:54:57.282 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T00:54:57.292 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:54:57.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:57.293 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T00:54:57.348 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:57.348 INFO:tasks.ceph:role: osd.5 2024-04-25T00:54:57.348 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:54:57.348 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T00:54:57.413 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:54:57.413 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:54:57.413 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:54:57.413 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:54:57.413 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:54:57.414 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:54:57.414 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:54:57.414 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:54:57.414 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:54:57.414 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:54:57.877 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T00:54:57.879 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T00:54:57.879 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T00:54:58.013 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T00:54:58.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:58.026 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:54:58.026 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T00:54:58.080 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:58.080 INFO:tasks.ceph:role: osd.6 2024-04-25T00:54:58.081 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:54:58.081 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T00:54:58.144 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:54:58.145 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:54:58.619 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T00:54:58.621 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T00:54:58.621 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T00:54:58.762 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T00:54:58.774 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:54:58.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:58.775 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T00:54:58.829 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T00:54:58.829 INFO:tasks.ceph:role: osd.7 2024-04-25T00:54:58.829 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:54:58.829 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T00:54:58.891 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T00:54:58.892 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T00:54:59.311 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2024-04-25T00:54:59.313 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2024-04-25T00:54:59.314 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T00:54:59.408 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T00:54:59.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:59.418 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T00:54:59.418 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:54:59.537 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:54:59.537+0000 7f4a3e7157c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T00:54:59.538 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:54:59.537+0000 7f4a3e7157c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T00:54:59.539 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:54:59.537+0000 7f4a3e7157c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:54:59.539 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:54:59.537+0000 7f4a3e7157c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:54:59.539 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:54:59.537+0000 7f4a3e7157c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T00:54:59.540 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:54:59.541+0000 7f4a3e7157c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T00:55:02.537 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T00:55:02.551 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:55:02.630 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:02.629+0000 7f2349c757c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T00:55:02.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:02.629+0000 7f2349c757c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T00:55:02.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:02.629+0000 7f2349c757c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:55:02.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:02.629+0000 7f2349c757c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:55:02.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:02.629+0000 7f2349c757c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T00:55:02.631 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:02.629+0000 7f2349c757c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T00:55:05.625 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T00:55:06.042 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:55:06.128 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:06.125+0000 7f06f32647c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T00:55:06.129 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:06.129+0000 7f06f32647c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T00:55:06.129 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:06.129+0000 7f06f32647c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:55:06.129 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:06.129+0000 7f06f32647c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:55:06.129 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:06.129+0000 7f06f32647c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T00:55:06.129 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:06.129+0000 7f06f32647c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T00:55:08.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:55:08.782 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:55:09.103 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T00:55:09.116 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:55:09.199 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:09.197+0000 7fceec3827c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T00:55:09.200 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:09.197+0000 7fceec3827c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T00:55:09.200 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:09.197+0000 7fceec3827c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:55:09.200 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:09.197+0000 7fceec3827c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:55:09.200 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:09.197+0000 7fceec3827c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T00:55:09.201 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:09.201+0000 7fceec3827c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T00:55:12.196 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T00:55:12.254 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T00:55:12.254 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:12.254 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T00:55:12.268 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:12.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T00:55:12.324 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:12.324 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T00:55:12.384 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:12.385 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T00:55:12.444 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:12.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T00:55:12.500 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:12.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T00:55:12.556 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:12.556 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T00:55:12.569 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:12.569 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T00:55:12.626 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:12.626 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T00:55:12.679 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:12.679 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T00:55:12.732 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:12.732 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T00:55:12.785 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:12.785 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T00:55:12.840 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:12.840 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T00:55:12.845 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T00:55:12.846 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T00:55:12.890 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T00:55:12.904 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2024-04-25T00:55:12.904 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDQqSlm0iw+ORAAmDIlUnXiJ58JXA14Mt1Qpg== 2024-04-25T00:55:12.904 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.a] 2024-04-25T00:55:12.904 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDRqSlmB1p7AxAApj8b+uJ+23NHlPP7yC8SJg== 2024-04-25T00:55:12.904 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2024-04-25T00:55:12.904 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDUqSlmNBbtEBAAJgEN6Jqa3Y70/wPnOMJcqA== 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDXqSlm8n6SFRAAQZXRwyE8uvokp9/lel0B9w== 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDaqSlmnbgHHhAA3pEOhNODws3ADApmcA02Dw== 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDdqSlmXGnOJBAAdlkLgvu931KjX6vM1rUN+A== 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDRqSlmviQvABAARqSTcVrjv+0YqfhTPI4UMg== 2024-04-25T00:55:12.905 INFO:teuthology.orchestra.run.smithi187.stdout:[mds.b] 2024-04-25T00:55:12.906 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDRqSlmgYqACRAA9nOmbvfzq2/LHofYEsUIyQ== 2024-04-25T00:55:12.906 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2024-04-25T00:55:12.906 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDjqSlmxA0mIBAASNsLaxsy+z4/T5zBzV6//Q== 2024-04-25T00:55:12.906 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2024-04-25T00:55:12.906 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDmqSlmH7SmJRAAj15G2CcndsUB+HV0OCS6kw== 2024-04-25T00:55:12.906 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2024-04-25T00:55:12.906 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDqqSlmr+i6BxAARbD2r40Zydtd1IkdC8RTVg== 2024-04-25T00:55:12.906 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2024-04-25T00:55:12.907 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDtqSlmcTMBDBAAIgIUHgsu/mBiq8QNBDDppw== 2024-04-25T00:55:12.907 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2024-04-25T00:55:12.907 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQDRqSlmLHGXDBAAcY5ixa+8Gv2MeY7OW7Jffw== 2024-04-25T00:55:12.907 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2024-04-25T00:55:12.907 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDQqSlm0iw+ORAAmDIlUnXiJ58JXA14Mt1Qpg== 2024-04-25T00:55:12.907 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2024-04-25T00:55:12.907 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDRqSlmB1p7AxAApj8b+uJ+23NHlPP7yC8SJg== 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDUqSlmNBbtEBAAJgEN6Jqa3Y70/wPnOMJcqA== 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDXqSlm8n6SFRAAQZXRwyE8uvokp9/lel0B9w== 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDaqSlmnbgHHhAA3pEOhNODws3ADApmcA02Dw== 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDdqSlmXGnOJBAAdlkLgvu931KjX6vM1rUN+A== 2024-04-25T00:55:12.908 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDRqSlmviQvABAARqSTcVrjv+0YqfhTPI4UMg== 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDRqSlmgYqACRAA9nOmbvfzq2/LHofYEsUIyQ== 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDjqSlmxA0mIBAASNsLaxsy+z4/T5zBzV6//Q== 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDmqSlmH7SmJRAAj15G2CcndsUB+HV0OCS6kw== 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2024-04-25T00:55:12.909 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDqqSlmr+i6BxAARbD2r40Zydtd1IkdC8RTVg== 2024-04-25T00:55:12.910 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2024-04-25T00:55:12.910 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDtqSlmcTMBDBAAIgIUHgsu/mBiq8QNBDDppw== 2024-04-25T00:55:12.910 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-04-25T00:55:12.910 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDRqSlmLHGXDBAAcY5ixa+8Gv2MeY7OW7Jffw== 2024-04-25T00:55:12.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:55:12.946 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:55:12.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:55:12.995 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:55:13.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.088 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.131 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.175 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.214 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.254 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.296 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.338 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.384 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:55:13.426 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T00:55:13.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:55:13.515 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T00:55:13.564 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.598 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.683 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.727 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.766 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.809 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.851 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T00:55:13.892 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:55:13.934 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T00:55:13.979 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T00:55:13.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T00:55:14.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:55:14.127 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T00:55:14.138 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T00:55:14.153 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:55:14.245 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T00:55:14.255 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T00:55:14.306 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T00:55:14.391 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T00:55:14.445 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:55:14.447 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T00:55:14.493 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T00:55:14.494 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T00:55:14.494 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T00:55:14.496 INFO:tasks.ceph.mon.a:Started 2024-04-25T00:55:14.497 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T00:55:14.497 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T00:55:14.538 INFO:tasks.ceph.mon.b:Started 2024-04-25T00:55:14.539 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T00:55:14.539 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T00:55:14.541 INFO:tasks.ceph.mon.c:Started 2024-04-25T00:55:14.542 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T00:55:14.542 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T00:55:14.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T00:55:14.545 INFO:tasks.ceph.mgr.y:Started 2024-04-25T00:55:14.545 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T00:55:14.545 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T00:55:14.548 INFO:tasks.ceph.mgr.x:Started 2024-04-25T00:55:14.548 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T00:55:14.548 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T00:55:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66f12b9640 1 Processor -- start 2024-04-25T00:55:14.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66f12b9640 1 -- start start 2024-04-25T00:55:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66f12b9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66f12b9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 0x7f66ec14fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66f12b9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66ec0729d0 con 0x7f66ec0725f0 2024-04-25T00:55:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66f12b9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec0b4f10 con 0x7f66ec0b5ad0 2024-04-25T00:55:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66f12b9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f66ec0b0720 con 0x7f66ec13f490 2024-04-25T00:55:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66ea7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66ea7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59266/0 (socket says 172.21.15.112:59266) 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66ea7fc640 1 -- 172.21.15.112:0/785228734 learned_addr learned my addr 172.21.15.112:0/785228734 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66ea7fc640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 msgr2=0x7f66ec13ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66ea7fc640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 msgr2=0x7f66ec13ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66eb7fe640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 0x7f66ec14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66eb7fe640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 msgr2=0x7f66ec14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66eb7fe640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 msgr2=0x7f66ec14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66eb7fe640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 0x7f66ec14fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:55:14.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.662+0000 7f66eb7fe640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 0x7f66ec14fbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66eb7fe640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 0x7f66ec14fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66ea7fc640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 msgr2=0x7f66ec13ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66ea7fc640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 msgr2=0x7f66ec13ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66eb7fe640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 msgr2=0x7f66ec14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66eb7fe640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 msgr2=0x7f66ec14fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66eb7fe640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 0x7f66ec14fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:55:14.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.866+0000 7f66eb7fe640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 0x7f66ec14fbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:55:14.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.898+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 264839931 0 0) 0x7f66ec0729d0 con 0x7f66ec0725f0 2024-04-25T00:55:14.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.898+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66d4003530 con 0x7f66ec0725f0 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 <== mon.1 v1:172.21.15.187:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 239682558 0 0) 0x7f66d4003530 con 0x7f66ec0725f0 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 msgr2=0x7f66ec14fbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.187:3300/0 conn(0x7f66ec13f490 0x7f66ec14fbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 msgr2=0x7f66ec13ef50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 --2- 172.21.15.112:0/785228734 >> v2:172.21.15.112:3300/0 conn(0x7f66ec0b5ad0 0x7f66ec13ef50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 --> v1:172.21.15.187:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec0b4ba0 con 0x7f66ec0725f0 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 <== mon.1 v1:172.21.15.187:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f66e0001680 con 0x7f66ec0725f0 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 >> v1:172.21.15.187:6789/0 conn(0x7f66ec0725f0 legacy=0x7f66ec0b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:14.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d40039c0 0x7f66d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:14.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4004810 0x7f66d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:14.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d40051d0 0x7f66d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:14.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b4f10 con 0x7f66d4004810 2024-04-25T00:55:14.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b0720 con 0x7f66d40051d0 2024-04-25T00:55:14.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f66d400c180 con 0x7f66d40039c0 2024-04-25T00:55:14.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4004810 0x7f66d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66eb7fe640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d40051d0 0x7f66d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66eaffd640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d40039c0 0x7f66d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66ea7fc640 1 -- 172.21.15.112:0/785228734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d40039c0 msgr2=0x7f66d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:14.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d40039c0 0x7f66d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:14.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66ea7fc640 1 -- 172.21.15.112:0/785228734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d40051d0 msgr2=0x7f66d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:14.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66d40051d0 0x7f66d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:14.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66ea7fc640 1 -- 172.21.15.112:0/785228734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d4003530 con 0x7f66d4004810 2024-04-25T00:55:14.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.906+0000 7f66eb7fe640 1 -- 172.21.15.112:0/785228734 reap_dead start 2024-04-25T00:55:14.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.910+0000 7f66ea7fc640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4004810 0x7f66d4004c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0096d0 tx=0x7f66dc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76327d0a00cf340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:14.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0ab070 con 0x7f66d4004810 2024-04-25T00:55:14.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66dc0a5a80 con 0x7f66d4004810 2024-04-25T00:55:14.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66e9ffb640 1 -- 172.21.15.112:0/785228734 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0098e0 con 0x7f66d4004810 2024-04-25T00:55:14.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 -- 172.21.15.112:0/785228734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4004810 msgr2=0x7f66d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:14.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4004810 0x7f66d4004c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0096d0 tx=0x7f66dc01cc80 comp rx=0 tx=0).stop 2024-04-25T00:55:14.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 -- 172.21.15.112:0/785228734 shutdown_connections 2024-04-25T00:55:14.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 --2- 172.21.15.112:0/785228734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66d4004810 0x7f66d4004c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:14.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 -- 172.21.15.112:0/785228734 >> 172.21.15.112:0/785228734 conn(0x7f66ec06d110 msgr2=0x7f66ec06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:14.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 -- 172.21.15.112:0/785228734 shutdown_connections 2024-04-25T00:55:14.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 -- 172.21.15.112:0/785228734 wait complete. 2024-04-25T00:55:14.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 Processor -- start 2024-04-25T00:55:14.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.914+0000 7f66f12b9640 1 -- start start 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66f12b9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d4004810 0x7f66ec13dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66f12b9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66ec145b60 0x7f66ec1de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66f12b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec1de820 0x7f66ec1e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66f12b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b5950 con 0x7f66ec1de820 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66f12b9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66ec0b57d0 con 0x7f66ec145b60 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66f12b9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f66ec13f8a0 con 0x7f66d4004810 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec1de820 0x7f66ec1e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66ea7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66ec145b60 0x7f66ec1de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eaffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d4004810 0x7f66ec13dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66ea7fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66ec145b60 0x7f66ec1de2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:53344/0 (socket says 172.21.15.112:53344) 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec1de820 0x7f66ec1e15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59322/0 (socket says 172.21.15.112:59322) 2024-04-25T00:55:14.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66ea7fc640 1 -- 172.21.15.112:0/1563365642 learned_addr learned my addr 172.21.15.112:0/1563365642 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:14.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eaffd640 1 -- 172.21.15.112:0/1563365642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66ec145b60 msgr2=0x7f66ec1de2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:14.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eaffd640 1 --2- 172.21.15.112:0/1563365642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66ec145b60 0x7f66ec1de2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:14.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eaffd640 1 -- 172.21.15.112:0/1563365642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec1de820 msgr2=0x7f66ec1e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:14.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eaffd640 1 --2- 172.21.15.112:0/1563365642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec1de820 0x7f66ec1e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:14.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eaffd640 1 -- 172.21.15.112:0/1563365642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66ec0b89e0 con 0x7f66d4004810 2024-04-25T00:55:14.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eb7fe640 1 --2- 172.21.15.112:0/1563365642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec1de820 0x7f66ec1e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:14.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.918+0000 7f66eaffd640 1 --2- 172.21.15.112:0/1563365642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d4004810 0x7f66ec13dab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f66e000f7b0 tx=0x7f66e000fc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:14.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.922+0000 7f66d37fe640 1 -- 172.21.15.112:0/1563365642 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e001a070 con 0x7f66d4004810 2024-04-25T00:55:14.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.922+0000 7f66d37fe640 1 -- 172.21.15.112:0/1563365642 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66e0005320 con 0x7f66d4004810 2024-04-25T00:55:14.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.922+0000 7f66d37fe640 1 -- 172.21.15.112:0/1563365642 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0007250 con 0x7f66d4004810 2024-04-25T00:55:14.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.922+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66ec144d80 con 0x7f66d4004810 2024-04-25T00:55:14.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.922+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f66ec145340 con 0x7f66d4004810 2024-04-25T00:55:14.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.922+0000 7f66d37fe640 1 -- 172.21.15.112:0/1563365642 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f66e00073f0 con 0x7f66d4004810 2024-04-25T00:55:14.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.922+0000 7f66d37fe640 1 -- 172.21.15.112:0/1563365642 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f66e000d990 con 0x7f66d4004810 2024-04-25T00:55:14.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.922+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66ec0b1220 con 0x7f66d4004810 2024-04-25T00:55:14.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.930+0000 7f66d37fe640 1 -- 172.21.15.112:0/1563365642 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f66e0023020 con 0x7f66d4004810 2024-04-25T00:55:14.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:14.994+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f66ec04f5f0 con 0x7f66d4004810 2024-04-25T00:55:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.002+0000 7f66d37fe640 1 -- 172.21.15.112:0/1563365642 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f66e000dbe0 con 0x7f66d4004810 2024-04-25T00:55:15.004 INFO:teuthology.orchestra.run.smithi112.stderr:adjusted tunables profile to default 2024-04-25T00:55:15.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d4004810 msgr2=0x7f66ec13dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 --2- 172.21.15.112:0/1563365642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d4004810 0x7f66ec13dab0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f66e000f7b0 tx=0x7f66e000fc70 comp rx=0 tx=0).stop 2024-04-25T00:55:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 shutdown_connections 2024-04-25T00:55:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 --2- 172.21.15.112:0/1563365642 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66ec1de820 0x7f66ec1e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 --2- 172.21.15.112:0/1563365642 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f66ec145b60 0x7f66ec1de2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 --2- 172.21.15.112:0/1563365642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66d4004810 0x7f66ec13dab0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 >> 172.21.15.112:0/1563365642 conn(0x7f66ec06d110 msgr2=0x7f66ec06d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 shutdown_connections 2024-04-25T00:55:15.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:15.006+0000 7f66f12b9640 1 -- 172.21.15.112:0/1563365642 wait complete. 2024-04-25T00:55:15.024 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T00:55:15.024 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T00:55:15.024 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:15.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T00:55:15.034 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:15.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T00:55:15.093 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:15.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T00:55:15.103 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:15.102+0000 7fd00f1a3180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:55:15.115 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:55:15.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T00:55:15.164 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:15.161+0000 7fb93e172180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:55:15.167 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:15.168 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T00:55:15.182 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:15.182 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T00:55:15.239 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:15.240 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T00:55:15.291 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2024-04-25T00:55:15.291 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T00:55:15.343 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 460ba393-4ad9-444f-bc8a-fafa413c291d 0 2024-04-25T00:55:15.506 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f4580ab0640 1 Processor -- start 2024-04-25T00:55:15.507 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f4580ab0640 1 -- start start 2024-04-25T00:55:15.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f4580ab0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f457c074530 0x7f457c144910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f4580ab0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c13e450 0x7f457c14ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.508 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f4580ab0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457c074910 con 0x7f457c0b2580 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f4580ab0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f457c0b5730 con 0x7f457c074530 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f4580ab0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f457c06fa90 con 0x7f457c13e450 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f457ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c13e450 0x7f457c14ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f457a59c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f457c074530 0x7f457c144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f457ad9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c13e450 0x7f457c14ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:59150/0 (socket says 172.21.15.187:59150) 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.505+0000 7f457ad9d640 1 -- 172.21.15.187:0/2711951250 learned_addr learned my addr 172.21.15.187:0/2711951250 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457ad9d640 1 -- 172.21.15.187:0/2711951250 >> v2:172.21.15.187:3300/0 conn(0x7f457c074530 msgr2=0x7f457c144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457ad9d640 1 --2- 172.21.15.187:0/2711951250 >> v2:172.21.15.187:3300/0 conn(0x7f457c074530 0x7f457c144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457ad9d640 1 -- 172.21.15.187:0/2711951250 >> v1:172.21.15.112:6789/0 conn(0x7f457c0b2580 legacy=0x7f457c13dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457ad9d640 1 -- 172.21.15.187:0/2711951250 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c0b2aa0 con 0x7f457c13e450 2024-04-25T00:55:15.509 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457a59c640 1 --2- 172.21.15.187:0/2711951250 >> v2:172.21.15.187:3300/0 conn(0x7f457c074530 0x7f457c144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:15.511 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457ad9d640 1 --2- 172.21.15.187:0/2711951250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c13e450 0x7f457c14ea00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f457400d150 tx=0x7f457401eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6abf0d039ee44340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:15.511 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457959a640 1 -- 172.21.15.187:0/2711951250 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457400b2d0 con 0x7f457c13e450 2024-04-25T00:55:15.511 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457959a640 1 -- 172.21.15.187:0/2711951250 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f457400b470 con 0x7f457c13e450 2024-04-25T00:55:15.511 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f457959a640 1 -- 172.21.15.187:0/2711951250 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45740a7aa0 con 0x7f457c13e450 2024-04-25T00:55:15.512 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f4580ab0640 1 -- 172.21.15.187:0/2711951250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c13e450 msgr2=0x7f457c14ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.512 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f4580ab0640 1 --2- 172.21.15.187:0/2711951250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c13e450 0x7f457c14ea00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f457400d150 tx=0x7f457401eba0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.512 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f4580ab0640 1 -- 172.21.15.187:0/2711951250 shutdown_connections 2024-04-25T00:55:15.512 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f4580ab0640 1 --2- 172.21.15.187:0/2711951250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c13e450 0x7f457c14ea00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.512 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f4580ab0640 1 --2- 172.21.15.187:0/2711951250 >> v2:172.21.15.187:3300/0 conn(0x7f457c074530 0x7f457c144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.512 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f4580ab0640 1 -- 172.21.15.187:0/2711951250 >> 172.21.15.187:0/2711951250 conn(0x7f457c06b810 msgr2=0x7f457c0b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:15.512 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f4580ab0640 1 -- 172.21.15.187:0/2711951250 shutdown_connections 2024-04-25T00:55:15.513 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.509+0000 7f4580ab0640 1 -- 172.21.15.187:0/2711951250 wait complete. 2024-04-25T00:55:15.513 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 Processor -- start 2024-04-25T00:55:15.514 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 -- start start 2024-04-25T00:55:15.516 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c074530 0x7f457c0b9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.516 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c0b2580 0x7f457c134600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.516 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457c13e450 0x7f457c134b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.516 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0b3420 con 0x7f457c13e450 2024-04-25T00:55:15.516 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0b32a0 con 0x7f457c074530 2024-04-25T00:55:15.516 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f457c144e50 con 0x7f457c0b2580 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c0b2580 0x7f457c134600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f457a59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c074530 0x7f457c0b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c0b2580 0x7f457c134600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:59164/0 (socket says 172.21.15.187:59164) 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 -- 172.21.15.187:0/1550084979 learned_addr learned my addr 172.21.15.187:0/1550084979 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f457ad9d640 1 --2- 172.21.15.187:0/1550084979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457c13e450 0x7f457c134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 -- 172.21.15.187:0/1550084979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c074530 msgr2=0x7f457c0b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 --2- 172.21.15.187:0/1550084979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c074530 0x7f457c0b9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 -- 172.21.15.187:0/1550084979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457c13e450 msgr2=0x7f457c134b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 --2- 172.21.15.187:0/1550084979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457c13e450 0x7f457c134b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 -- 172.21.15.187:0/1550084979 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c141bd0 con 0x7f457c0b2580 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4579d9b640 1 --2- 172.21.15.187:0/1550084979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c0b2580 0x7f457c134600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f456400c420 tx=0x7f456400c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4572ffd640 1 -- 172.21.15.187:0/1550084979 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45640052d0 con 0x7f457c0b2580 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4572ffd640 1 -- 172.21.15.187:0/1550084979 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4564005470 con 0x7f457c0b2580 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c0b8df0 con 0x7f457c0b2580 2024-04-25T00:55:15.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4572ffd640 1 -- 172.21.15.187:0/1550084979 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564007260 con 0x7f457c0b2580 2024-04-25T00:55:15.518 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c137140 con 0x7f457c0b2580 2024-04-25T00:55:15.518 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4572ffd640 1 -- 172.21.15.187:0/1550084979 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4564007400 con 0x7f457c0b2580 2024-04-25T00:55:15.518 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.513+0000 7f4572ffd640 1 -- 172.21.15.187:0/1550084979 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f4564003ad0 con 0x7f457c0b2580 2024-04-25T00:55:15.518 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.517+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457c141dd0 con 0x7f457c0b2580 2024-04-25T00:55:15.518 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.517+0000 7f4572ffd640 1 -- 172.21.15.187:0/1550084979 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f457c141dd0 con 0x7f457c0b2580 2024-04-25T00:55:15.579 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:15.578+0000 7fd00f1a3180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:55:15.587 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.585+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "460ba393-4ad9-444f-bc8a-fafa413c291d", "id": 0} v 0) -- 0x7f457c074b60 con 0x7f457c0b2580 2024-04-25T00:55:15.594 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4572ffd640 1 -- 172.21.15.187:0/1550084979 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "460ba393-4ad9-444f-bc8a-fafa413c291d", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f4564003d20 con 0x7f457c0b2580 2024-04-25T00:55:15.594 INFO:teuthology.orchestra.run.smithi187.stdout:0 2024-04-25T00:55:15.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c0b2580 msgr2=0x7f457c134600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 --2- 172.21.15.187:0/1550084979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c0b2580 0x7f457c134600 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f456400c420 tx=0x7f456400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 shutdown_connections 2024-04-25T00:55:15.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 --2- 172.21.15.187:0/1550084979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f457c13e450 0x7f457c134b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 --2- 172.21.15.187:0/1550084979 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f457c0b2580 0x7f457c134600 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 --2- 172.21.15.187:0/1550084979 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f457c074530 0x7f457c0b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.596 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 >> 172.21.15.187:0/1550084979 conn(0x7f457c06b810 msgr2=0x7f457c069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:15.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 shutdown_connections 2024-04-25T00:55:15.597 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.593+0000 7f4580ab0640 1 -- 172.21.15.187:0/1550084979 wait complete. 2024-04-25T00:55:15.612 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 07888c5e-f032-433a-9294-06a2adec15bf 1 2024-04-25T00:55:15.661 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:15.661+0000 7fb93e172180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:55:15.727 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.725+0000 7f08a5b48640 1 Processor -- start 2024-04-25T00:55:15.727 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.725+0000 7f08a5b48640 1 -- start start 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.725+0000 7f08a5b48640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f08a0148860 0x7f08a00b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.725+0000 7f08a5b48640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08a014dbd0 0x7f08a014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.725+0000 7f08a5b48640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08a0072350 con 0x7f08a00b5600 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.725+0000 7f08a5b48640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f08a0072630 con 0x7f08a0148860 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.725+0000 7f08a5b48640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f08a0072910 con 0x7f08a014dbd0 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08a014dbd0 0x7f08a014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089f7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f08a00b5600 0x7f08a00b2e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:56942/0 (socket says 172.21.15.187:56942) 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089f7fe640 1 -- 172.21.15.187:0/3586199246 learned_addr learned my addr 172.21.15.187:0/3586199246 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T00:55:15.729 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089effd640 1 --2- 172.21.15.187:0/3586199246 >> v2:172.21.15.112:3300/0 conn(0x7f08a0148860 0x7f08a00b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.730 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089ffff640 1 -- 172.21.15.187:0/3586199246 >> v2:172.21.15.112:3300/0 conn(0x7f08a0148860 msgr2=0x7f08a00b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:15.730 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089ffff640 1 --2- 172.21.15.187:0/3586199246 >> v2:172.21.15.112:3300/0 conn(0x7f08a0148860 0x7f08a00b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.730 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089ffff640 1 -- 172.21.15.187:0/3586199246 >> v1:172.21.15.187:6789/0 conn(0x7f08a00b5600 legacy=0x7f08a00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.730 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089e7fc640 1 -- 172.21.15.187:0/3586199246 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3903650085 0 0) 0x7f08a0072350 con 0x7f08a00b5600 2024-04-25T00:55:15.730 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089ffff640 1 -- 172.21.15.187:0/3586199246 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a00b3d60 con 0x7f08a014dbd0 2024-04-25T00:55:15.732 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089ffff640 1 --2- 172.21.15.187:0/3586199246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a014dbd0 0x7f08a014ffc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f08a006c150 tx=0x7f089801c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c6446f08585704 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:15.732 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089e7fc640 1 -- 172.21.15.187:0/3586199246 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980b2070 con 0x7f08a014dbd0 2024-04-25T00:55:15.732 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089e7fc640 1 -- 172.21.15.187:0/3586199246 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08980a5870 con 0x7f08a014dbd0 2024-04-25T00:55:15.732 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.729+0000 7f089e7fc640 1 -- 172.21.15.187:0/3586199246 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08980a5c40 con 0x7f08a014dbd0 2024-04-25T00:55:15.733 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- 172.21.15.187:0/3586199246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a014dbd0 msgr2=0x7f08a014ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.733 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 --2- 172.21.15.187:0/3586199246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a014dbd0 0x7f08a014ffc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f08a006c150 tx=0x7f089801c8a0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.733 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- 172.21.15.187:0/3586199246 shutdown_connections 2024-04-25T00:55:15.733 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 --2- 172.21.15.187:0/3586199246 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a014dbd0 0x7f08a014ffc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.733 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 --2- 172.21.15.187:0/3586199246 >> v2:172.21.15.112:3300/0 conn(0x7f08a0148860 0x7f08a00b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.733 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- 172.21.15.187:0/3586199246 >> 172.21.15.187:0/3586199246 conn(0x7f08a00b6440 msgr2=0x7f08a00b6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:15.733 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- 172.21.15.187:0/3586199246 shutdown_connections 2024-04-25T00:55:15.734 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- 172.21.15.187:0/3586199246 wait complete. 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 Processor -- start 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- start start 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a00b5600 0x7f08a0071850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a0148860 0x7f08a01e32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a014dbd0 0x7f08a01e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08a00b7200 con 0x7f08a00b5600 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08a0148c70 con 0x7f08a0148860 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08a00b7380 con 0x7f08a014dbd0 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a0148860 0x7f08a01e32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a014dbd0 0x7f08a01e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a0148860 0x7f08a01e32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:59986/0 (socket says 172.21.15.187:59986) 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 -- 172.21.15.187:0/1421959223 learned_addr learned my addr 172.21.15.187:0/1421959223 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089f7fe640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a00b5600 0x7f08a0071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 -- 172.21.15.187:0/1421959223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a014dbd0 msgr2=0x7f08a01e37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a014dbd0 0x7f08a01e37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 -- 172.21.15.187:0/1421959223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a00b5600 msgr2=0x7f08a0071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a00b5600 0x7f08a0071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 -- 172.21.15.187:0/1421959223 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a014e570 con 0x7f08a0148860 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089ffff640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a014dbd0 0x7f08a01e37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f089effd640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a0148860 0x7f08a01e32a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f089400d800 tx=0x7f089400dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f0887fff640 1 -- 172.21.15.187:0/1421959223 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089400b6c0 con 0x7f08a0148860 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f0887fff640 1 -- 172.21.15.187:0/1421959223 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0894019070 con 0x7f08a0148860 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f0887fff640 1 -- 172.21.15.187:0/1421959223 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089400ba60 con 0x7f08a0148860 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a0147660 con 0x7f08a0148860 2024-04-25T00:55:15.737 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a00b8a60 con 0x7f08a0148860 2024-04-25T00:55:15.738 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f0887fff640 1 -- 172.21.15.187:0/1421959223 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f089400bc00 con 0x7f08a0148860 2024-04-25T00:55:15.738 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.733+0000 7f0887fff640 1 -- 172.21.15.187:0/1421959223 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f0894014950 con 0x7f08a0148860 2024-04-25T00:55:15.738 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.737+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a0074b50 con 0x7f08a0148860 2024-04-25T00:55:15.739 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.737+0000 7f0887fff640 1 -- 172.21.15.187:0/1421959223 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0894014c00 con 0x7f08a0148860 2024-04-25T00:55:15.773 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:15.774+0000 7fd00f1a3180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:55:15.808 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.805+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "07888c5e-f032-433a-9294-06a2adec15bf", "id": 1} v 0) -- 0x7f08a00b2a50 con 0x7f08a0148860 2024-04-25T00:55:15.813 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f0887fff640 1 -- 172.21.15.187:0/1421959223 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07888c5e-f032-433a-9294-06a2adec15bf", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f089401e460 con 0x7f08a0148860 2024-04-25T00:55:15.813 INFO:teuthology.orchestra.run.smithi187.stdout:1 2024-04-25T00:55:15.814 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a0148860 msgr2=0x7f08a01e32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.814 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a0148860 0x7f08a01e32a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f089400d800 tx=0x7f089400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.814 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 shutdown_connections 2024-04-25T00:55:15.814 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08a014dbd0 0x7f08a01e37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08a0148860 0x7f08a01e32a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 --2- 172.21.15.187:0/1421959223 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08a00b5600 0x7f08a0071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 >> 172.21.15.187:0/1421959223 conn(0x7f08a00b6440 msgr2=0x7f08a01398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:15.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 shutdown_connections 2024-04-25T00:55:15.815 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.813+0000 7f08a5b48640 1 -- 172.21.15.187:0/1421959223 wait complete. 2024-04-25T00:55:15.830 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 73523f2d-5c2d-44dd-9227-05493e90372d 2 2024-04-25T00:55:15.874 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:15.873+0000 7fb93e172180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:55:15.947 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f0336f17640 1 Processor -- start 2024-04-25T00:55:15.947 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f0336f17640 1 -- start start 2024-04-25T00:55:15.948 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f0336f17640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0330074530 0x7f0330144660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.948 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f0336f17640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0330144ba0 0x7f033013ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.948 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f0336f17640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f033013e7a0 con 0x7f033013e310 2024-04-25T00:55:15.948 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f0336f17640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0330145000 con 0x7f0330074530 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f0336f17640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f033006fa50 con 0x7f0330144ba0 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f032ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0330144ba0 0x7f033013ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f03354b4640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f033013e310 0x7f0330147600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:56952/0 (socket says 172.21.15.187:56952) 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f03354b4640 1 -- 172.21.15.187:0/1009561985 learned_addr learned my addr 172.21.15.187:0/1009561985 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f0334cb3640 1 --2- 172.21.15.187:0/1009561985 >> v2:172.21.15.112:3300/0 conn(0x7f0330074530 0x7f0330144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f032ffff640 1 -- 172.21.15.187:0/1009561985 >> v2:172.21.15.112:3300/0 conn(0x7f0330074530 msgr2=0x7f0330144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f032ffff640 1 --2- 172.21.15.187:0/1009561985 >> v2:172.21.15.112:3300/0 conn(0x7f0330074530 0x7f0330144660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f032ffff640 1 -- 172.21.15.187:0/1009561985 >> v1:172.21.15.187:6789/0 conn(0x7f033013e310 legacy=0x7f0330147600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f032ffff640 1 -- 172.21.15.187:0/1009561985 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0330074910 con 0x7f0330144ba0 2024-04-25T00:55:15.949 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.945+0000 7f032f7fe640 1 -- 172.21.15.187:0/1009561985 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2391181211 0 0) 0x7f033013e7a0 con 0x7f033013e310 2024-04-25T00:55:15.950 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f032ffff640 1 --2- 172.21.15.187:0/1009561985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0330144ba0 0x7f033013ddd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03240097f0 tx=0x7f032401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91de11aecd400ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:15.950 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f032f7fe640 1 -- 172.21.15.187:0/1009561985 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240b0070 con 0x7f0330144ba0 2024-04-25T00:55:15.950 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f032f7fe640 1 -- 172.21.15.187:0/1009561985 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03240ab600 con 0x7f0330144ba0 2024-04-25T00:55:15.950 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f032f7fe640 1 -- 172.21.15.187:0/1009561985 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240ab9d0 con 0x7f0330144ba0 2024-04-25T00:55:15.951 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 -- 172.21.15.187:0/1009561985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0330144ba0 msgr2=0x7f033013ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.951 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 --2- 172.21.15.187:0/1009561985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0330144ba0 0x7f033013ddd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03240097f0 tx=0x7f032401c3e0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.952 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 -- 172.21.15.187:0/1009561985 shutdown_connections 2024-04-25T00:55:15.952 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 --2- 172.21.15.187:0/1009561985 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0330144ba0 0x7f033013ddd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.952 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 --2- 172.21.15.187:0/1009561985 >> v2:172.21.15.112:3300/0 conn(0x7f0330074530 0x7f0330144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.952 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 -- 172.21.15.187:0/1009561985 >> 172.21.15.187:0/1009561985 conn(0x7f033006b810 msgr2=0x7f03300b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:15.952 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 -- 172.21.15.187:0/1009561985 shutdown_connections 2024-04-25T00:55:15.952 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 -- 172.21.15.187:0/1009561985 wait complete. 2024-04-25T00:55:15.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 Processor -- start 2024-04-25T00:55:15.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.949+0000 7f0336f17640 1 -- start start 2024-04-25T00:55:15.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0336f17640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330074530 0x7f03300b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.954 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0336f17640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033013e310 0x7f03301dd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0336f17640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330144ba0 0x7f0330135320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0336f17640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f03300b56e0 con 0x7f0330074530 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0336f17640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f03300b3530 con 0x7f0330144ba0 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0336f17640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03300b2bb0 con 0x7f033013e310 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033013e310 0x7f03301dd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f03354b4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330144ba0 0x7f0330135320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033013e310 0x7f03301dd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:59178/0 (socket says 172.21.15.187:59178) 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 -- 172.21.15.187:0/324988164 learned_addr learned my addr 172.21.15.187:0/324988164 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0334cb3640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330074530 0x7f03300b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 -- 172.21.15.187:0/324988164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330144ba0 msgr2=0x7f0330135320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330144ba0 0x7f0330135320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 -- 172.21.15.187:0/324988164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330074530 msgr2=0x7f03300b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330074530 0x7f03300b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:15.955 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 -- 172.21.15.187:0/324988164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033013ff30 con 0x7f033013e310 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f03354b4640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330144ba0 0x7f0330135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032ffff640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033013e310 0x7f03301dd830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f032401a010 tx=0x7f0324019180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032d7fa640 1 -- 172.21.15.187:0/324988164 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240b8020 con 0x7f033013e310 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032d7fa640 1 -- 172.21.15.187:0/324988164 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03240b4410 con 0x7f033013e310 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032d7fa640 1 -- 172.21.15.187:0/324988164 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03240b0040 con 0x7f033013e310 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0336f17640 1 -- 172.21.15.187:0/324988164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033013fc50 con 0x7f033013e310 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0336f17640 1 -- 172.21.15.187:0/324988164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301ddd70 con 0x7f033013e310 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032d7fa640 1 -- 172.21.15.187:0/324988164 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f03240bb020 con 0x7f033013e310 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f032d7fa640 1 -- 172.21.15.187:0/324988164 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f03240b4b90 con 0x7f033013e310 2024-04-25T00:55:15.956 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.953+0000 7f0316ffd640 1 -- 172.21.15.187:0/324988164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03080052a0 con 0x7f033013e310 2024-04-25T00:55:15.957 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:15.957+0000 7f032d7fa640 1 -- 172.21.15.187:0/324988164 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f03240c2030 con 0x7f033013e310 2024-04-25T00:55:16.026 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.025+0000 7f0316ffd640 1 -- 172.21.15.187:0/324988164 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "73523f2d-5c2d-44dd-9227-05493e90372d", "id": 2} v 0) -- 0x7f0308005860 con 0x7f033013e310 2024-04-25T00:55:16.032 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.029+0000 7f032d7fa640 1 -- 172.21.15.187:0/324988164 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73523f2d-5c2d-44dd-9227-05493e90372d", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0324016090 con 0x7f033013e310 2024-04-25T00:55:16.033 INFO:teuthology.orchestra.run.smithi187.stdout:2 2024-04-25T00:55:16.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 -- 172.21.15.187:0/324988164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033013e310 msgr2=0x7f03301dd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033013e310 0x7f03301dd830 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f032401a010 tx=0x7f0324019180 comp rx=0 tx=0).stop 2024-04-25T00:55:16.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 -- 172.21.15.187:0/324988164 shutdown_connections 2024-04-25T00:55:16.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0330144ba0 0x7f0330135320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f033013e310 0x7f03301dd830 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 --2- 172.21.15.187:0/324988164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0330074530 0x7f03300b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 -- 172.21.15.187:0/324988164 >> 172.21.15.187:0/324988164 conn(0x7f033006b810 msgr2=0x7f03301422e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.035 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 -- 172.21.15.187:0/324988164 shutdown_connections 2024-04-25T00:55:16.036 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.033+0000 7f0316ffd640 1 -- 172.21.15.187:0/324988164 wait complete. 2024-04-25T00:55:16.049 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new bf987335-4191-4de3-ac02-490605f11e83 3 2024-04-25T00:55:16.211 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.209+0000 7fe8b9ffe640 1 Processor -- start 2024-04-25T00:55:16.211 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.209+0000 7fe8b9ffe640 1 -- start start 2024-04-25T00:55:16.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.209+0000 7fe8b9ffe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe8b4148ad0 0x7fe8b4148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.209+0000 7fe8b9ffe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b40b7130 0x7fe8b41493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.213 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.209+0000 7fe8b9ffe640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8b406f370 con 0x7fe8b406ee70 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.209+0000 7fe8b9ffe640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe8b413fac0 con 0x7fe8b4148ad0 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.209+0000 7fe8b9ffe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b406f6c0 con 0x7fe8b40b7130 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b2ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b40b7130 0x7fe8b41493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b3fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fe8b406ee70 0x7fe8b40b4410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:56960/0 (socket says 172.21.15.187:56960) 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b3fff640 1 -- 172.21.15.187:0/3889759774 learned_addr learned my addr 172.21.15.187:0/3889759774 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b37fe640 1 --2- 172.21.15.187:0/3889759774 >> v2:172.21.15.112:3300/0 conn(0x7fe8b4148ad0 0x7fe8b4148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b27fc640 1 -- 172.21.15.187:0/3889759774 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1822177700 0 0) 0x7fe8b406f370 con 0x7fe8b406ee70 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b2ffd640 1 -- 172.21.15.187:0/3889759774 >> v2:172.21.15.112:3300/0 conn(0x7fe8b4148ad0 msgr2=0x7fe8b4148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b2ffd640 1 --2- 172.21.15.187:0/3889759774 >> v2:172.21.15.112:3300/0 conn(0x7fe8b4148ad0 0x7fe8b4148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b2ffd640 1 -- 172.21.15.187:0/3889759774 >> v1:172.21.15.187:6789/0 conn(0x7fe8b406ee70 legacy=0x7fe8b40b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.214 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b2ffd640 1 -- 172.21.15.187:0/3889759774 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b406e390 con 0x7fe8b40b7130 2024-04-25T00:55:16.215 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b2ffd640 1 --2- 172.21.15.187:0/3889759774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b40b7130 0x7fe8b41493f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80097c0 tx=0x7fe8a801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2feb5b9449e62ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:16.215 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b27fc640 1 -- 172.21.15.187:0/3889759774 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80b0070 con 0x7fe8b40b7130 2024-04-25T00:55:16.215 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b27fc640 1 -- 172.21.15.187:0/3889759774 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a80ab600 con 0x7fe8b40b7130 2024-04-25T00:55:16.216 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.213+0000 7fe8b27fc640 1 -- 172.21.15.187:0/3889759774 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80ab9d0 con 0x7fe8b40b7130 2024-04-25T00:55:16.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/3889759774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b40b7130 msgr2=0x7fe8b41493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 --2- 172.21.15.187:0/3889759774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b40b7130 0x7fe8b41493f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80097c0 tx=0x7fe8a801c410 comp rx=0 tx=0).stop 2024-04-25T00:55:16.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/3889759774 shutdown_connections 2024-04-25T00:55:16.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 --2- 172.21.15.187:0/3889759774 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b40b7130 0x7fe8b41493f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 --2- 172.21.15.187:0/3889759774 >> v2:172.21.15.112:3300/0 conn(0x7fe8b4148ad0 0x7fe8b4148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/3889759774 >> 172.21.15.187:0/3889759774 conn(0x7fe8b40711f0 msgr2=0x7fe8b4071600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/3889759774 shutdown_connections 2024-04-25T00:55:16.217 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/3889759774 wait complete. 2024-04-25T00:55:16.219 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 Processor -- start 2024-04-25T00:55:16.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- start start 2024-04-25T00:55:16.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b406ee70 0x7fe8b4149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b40b7130 0x7fe8b41426f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b4148ad0 0x7fe8b414a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b4152540 con 0x7fe8b4148ad0 2024-04-25T00:55:16.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b41523c0 con 0x7fe8b40b7130 2024-04-25T00:55:16.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b41526c0 con 0x7fe8b406ee70 2024-04-25T00:55:16.220 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b37fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b406ee70 0x7fe8b4149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b40b7130 0x7fe8b41426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b37fe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b406ee70 0x7fe8b4149230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:59194/0 (socket says 172.21.15.187:59194) 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b2ffd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b40b7130 0x7fe8b41426f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:60008/0 (socket says 172.21.15.187:60008) 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b37fe640 1 -- 172.21.15.187:0/536241817 learned_addr learned my addr 172.21.15.187:0/536241817 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b3fff640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b4148ad0 0x7fe8b414a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b2ffd640 1 -- 172.21.15.187:0/536241817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b406ee70 msgr2=0x7fe8b4149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b2ffd640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b406ee70 0x7fe8b4149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b2ffd640 1 -- 172.21.15.187:0/536241817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b4148ad0 msgr2=0x7fe8b414a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b2ffd640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b4148ad0 0x7fe8b414a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b2ffd640 1 -- 172.21.15.187:0/536241817 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b4141610 con 0x7fe8b40b7130 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b37fe640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b406ee70 0x7fe8b4149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b2ffd640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b40b7130 0x7fe8b41426f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80ab000 tx=0x7fe8a80191b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe89bfff640 1 -- 172.21.15.187:0/536241817 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80b8020 con 0x7fe8b40b7130 2024-04-25T00:55:16.221 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe89bfff640 1 -- 172.21.15.187:0/536241817 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8a80b4610 con 0x7fe8b40b7130 2024-04-25T00:55:16.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b414ad70 con 0x7fe8b40b7130 2024-04-25T00:55:16.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe89bfff640 1 -- 172.21.15.187:0/536241817 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80b0040 con 0x7fe8b40b7130 2024-04-25T00:55:16.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b4143000 con 0x7fe8b40b7130 2024-04-25T00:55:16.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe89bfff640 1 -- 172.21.15.187:0/536241817 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8a80bb020 con 0x7fe8b40b7130 2024-04-25T00:55:16.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe89bfff640 1 -- 172.21.15.187:0/536241817 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe8a80aa3a0 con 0x7fe8b40b7130 2024-04-25T00:55:16.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.217+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b4068470 con 0x7fe8b40b7130 2024-04-25T00:55:16.222 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.221+0000 7fe89bfff640 1 -- 172.21.15.187:0/536241817 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe8a80c2030 con 0x7fe8b40b7130 2024-04-25T00:55:16.290 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.289+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bf987335-4191-4de3-ac02-490605f11e83", "id": 3} v 0) -- 0x7fe8b4072800 con 0x7fe8b40b7130 2024-04-25T00:55:16.296 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.293+0000 7fe89bfff640 1 -- 172.21.15.187:0/536241817 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bf987335-4191-4de3-ac02-490605f11e83", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe8a8016090 con 0x7fe8b40b7130 2024-04-25T00:55:16.296 INFO:teuthology.orchestra.run.smithi187.stdout:3 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b40b7130 msgr2=0x7fe8b41426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b40b7130 0x7fe8b41426f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80ab000 tx=0x7fe8a80191b0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 shutdown_connections 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe8b4148ad0 0x7fe8b414a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe8b40b7130 0x7fe8b41426f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 --2- 172.21.15.187:0/536241817 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe8b406ee70 0x7fe8b4149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 >> 172.21.15.187:0/536241817 conn(0x7fe8b40711f0 msgr2=0x7fe8b40715d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 shutdown_connections 2024-04-25T00:55:16.298 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.297+0000 7fe8b9ffe640 1 -- 172.21.15.187:0/536241817 wait complete. 2024-04-25T00:55:16.312 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.310+0000 7fd00f1a3180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:55:16.314 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 56e4594f-d522-4bb1-985c-b308abc0e315 4 2024-04-25T00:55:16.365 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.366+0000 7fd00f1a3180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:55:16.418 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.414+0000 7fd00f1a3180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:55:16.442 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fe1108640 1 Processor -- start 2024-04-25T00:55:16.442 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fe1108640 1 -- start start 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fe1108640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9fdc0b68b0 0x7f9fdc14aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fe1108640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9fdc14fd80 0x7f9fdc152170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fe1108640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fdc0b6c90 con 0x7f9fdc0723d0 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fe1108640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9fdc13e300 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fe1108640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9fdc072830 con 0x7f9fdc14fd80 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdb59e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9fdc14fd80 0x7f9fdc152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdb59e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9fdc14fd80 0x7f9fdc152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:60016/0 (socket says 172.21.15.187:60016) 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdb59e640 1 -- 172.21.15.187:0/18555630 learned_addr learned my addr 172.21.15.187:0/18555630 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdad9d640 1 --2- 172.21.15.187:0/18555630 >> v2:172.21.15.112:3300/0 conn(0x7f9fdc0b68b0 0x7f9fdc14aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdb59e640 1 -- 172.21.15.187:0/18555630 >> v2:172.21.15.112:3300/0 conn(0x7f9fdc0b68b0 msgr2=0x7f9fdc14aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdb59e640 1 --2- 172.21.15.187:0/18555630 >> v2:172.21.15.112:3300/0 conn(0x7f9fdc0b68b0 0x7f9fdc14aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdb59e640 1 -- 172.21.15.187:0/18555630 >> v1:172.21.15.112:6789/0 conn(0x7f9fdc0723d0 legacy=0x7f9fdc13dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdb59e640 1 -- 172.21.15.187:0/18555630 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc0b5fb0 con 0x7f9fdc14fd80 2024-04-25T00:55:16.443 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdad9d640 1 --2- 172.21.15.187:0/18555630 >> v2:172.21.15.112:3300/0 conn(0x7f9fdc0b68b0 0x7f9fdc14aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fdb59e640 1 --2- 172.21.15.187:0/18555630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc14fd80 0x7f9fdc152170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009e50 tx=0x7f9fd401c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b8c7bb8da205e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fd9d9b640 1 -- 172.21.15.187:0/18555630 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40a5670 con 0x7f9fdc14fd80 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fd9d9b640 1 -- 172.21.15.187:0/18555630 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd40a5810 con 0x7f9fdc14fd80 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.433+0000 7f9fd9d9b640 1 -- 172.21.15.187:0/18555630 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40a5be0 con 0x7f9fdc14fd80 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- 172.21.15.187:0/18555630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc14fd80 msgr2=0x7f9fdc152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 --2- 172.21.15.187:0/18555630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc14fd80 0x7f9fdc152170 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009e50 tx=0x7f9fd401c9b0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- 172.21.15.187:0/18555630 shutdown_connections 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 --2- 172.21.15.187:0/18555630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc14fd80 0x7f9fdc152170 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 --2- 172.21.15.187:0/18555630 >> v2:172.21.15.112:3300/0 conn(0x7f9fdc0b68b0 0x7f9fdc14aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- 172.21.15.187:0/18555630 >> 172.21.15.187:0/18555630 conn(0x7f9fdc06d110 msgr2=0x7f9fdc06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- 172.21.15.187:0/18555630 shutdown_connections 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- 172.21.15.187:0/18555630 wait complete. 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 Processor -- start 2024-04-25T00:55:16.444 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- start start 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fdc0723d0 0x7f9fdc149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc0b68b0 0x7f9fdc1e0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fdc14fd80 0x7f9fdc1d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc1419d0 con 0x7f9fdc0723d0 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9fdc072c00 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9fdc141b50 con 0x7f9fdc14fd80 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc0b68b0 0x7f9fdc1e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fdb59e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fdc14fd80 0x7f9fdc1d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc0b68b0 0x7f9fdc1e0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:60020/0 (socket says 172.21.15.187:60020) 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 -- 172.21.15.187:0/2590310492 learned_addr learned my addr 172.21.15.187:0/2590310492 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fdad9d640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fdc0723d0 0x7f9fdc149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 -- 172.21.15.187:0/2590310492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fdc14fd80 msgr2=0x7f9fdc1d9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fdc14fd80 0x7f9fdc1d9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 -- 172.21.15.187:0/2590310492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fdc0723d0 msgr2=0x7f9fdc149880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.445 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fdc0723d0 0x7f9fdc149880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 -- 172.21.15.187:0/2590310492 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fdc1e3920 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fdad9d640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fdc0723d0 0x7f9fdc149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fda59c640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc0b68b0 0x7f9fdc1e0280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000c4b0 tx=0x7f9fd000c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fc37fe640 1 -- 172.21.15.187:0/2590310492 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0012830 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fc37fe640 1 -- 172.21.15.187:0/2590310492 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fd0017070 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fdc1e3bc0 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fc37fe640 1 -- 172.21.15.187:0/2590310492 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0012c30 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fdc1416e0 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fc37fe640 1 -- 172.21.15.187:0/2590310492 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9fd001c460 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.437+0000 7f9fc37fe640 1 -- 172.21.15.187:0/2590310492 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f9fd001bc40 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.446 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.441+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fdc04f5f0 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.447 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.441+0000 7f9fc37fe640 1 -- 172.21.15.187:0/2590310492 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9fd0012dd0 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.447 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:16.441+0000 7fb93e172180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:55:16.492 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.490+0000 7fd00f1a3180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:55:16.498 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:16.497+0000 7fb93e172180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:55:16.511 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.509+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "56e4594f-d522-4bb1-985c-b308abc0e315", "id": 4} v 0) -- 0x7f9fdc0b3dd0 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.517 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fc37fe640 1 -- 172.21.15.187:0/2590310492 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "56e4594f-d522-4bb1-985c-b308abc0e315", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f9fd001b730 con 0x7f9fdc0b68b0 2024-04-25T00:55:16.517 INFO:teuthology.orchestra.run.smithi187.stdout:4 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc0b68b0 msgr2=0x7f9fdc1e0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc0b68b0 0x7f9fdc1e0280 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000c4b0 tx=0x7f9fd000c970 comp rx=0 tx=0).stop 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 shutdown_connections 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9fdc14fd80 0x7f9fdc1d9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9fdc0b68b0 0x7f9fdc1e0280 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 --2- 172.21.15.187:0/2590310492 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9fdc0723d0 0x7f9fdc149880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 >> 172.21.15.187:0/2590310492 conn(0x7f9fdc06d110 msgr2=0x7f9fdc0b99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 shutdown_connections 2024-04-25T00:55:16.519 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.517+0000 7f9fe1108640 1 -- 172.21.15.187:0/2590310492 wait complete. 2024-04-25T00:55:16.534 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 58cf3cc8-e445-4def-b50a-f502aa6a24c7 5 2024-04-25T00:55:16.545 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.546+0000 7fd00f1a3180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:55:16.555 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:16.553+0000 7fb93e172180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:55:16.620 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.618+0000 7fd00f1a3180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:55:16.635 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:16.633+0000 7fb93e172180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:55:16.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82cd674640 1 Processor -- start 2024-04-25T00:55:16.651 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82cd674640 1 -- start start 2024-04-25T00:55:16.656 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82cd674640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82c8074530 0x7f82c80ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.656 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82cd674640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f82c80ae850 0x7f82c814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.657 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82cd674640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82c80afbd0 con 0x7f82c814af70 2024-04-25T00:55:16.657 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82cd674640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f82c80b5900 con 0x7f82c80ae850 2024-04-25T00:55:16.657 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82cd674640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f82c80afe40 con 0x7f82c8074530 2024-04-25T00:55:16.657 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82c6ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82c8074530 0x7f82c80ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.658 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82c6ffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f82c8074530 0x7f82c80ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:60030/0 (socket says 172.21.15.187:60030) 2024-04-25T00:55:16.658 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82c6ffd640 1 -- 172.21.15.187:0/2739370506 learned_addr learned my addr 172.21.15.187:0/2739370506 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:16.658 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82c67fc640 1 --2- 172.21.15.187:0/2739370506 >> v2:172.21.15.112:3300/0 conn(0x7f82c80ae850 0x7f82c814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.658 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82c6ffd640 1 -- 172.21.15.187:0/2739370506 >> v2:172.21.15.112:3300/0 conn(0x7f82c80ae850 msgr2=0x7f82c814aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:16.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82c6ffd640 1 --2- 172.21.15.187:0/2739370506 >> v2:172.21.15.112:3300/0 conn(0x7f82c80ae850 0x7f82c814aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82c6ffd640 1 -- 172.21.15.187:0/2739370506 >> v1:172.21.15.112:6789/0 conn(0x7f82c814af70 legacy=0x7f82c81440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:16.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.649+0000 7f82c6ffd640 1 -- 172.21.15.187:0/2739370506 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c80aecc0 con 0x7f82c8074530 2024-04-25T00:55:16.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82c6ffd640 1 --2- 172.21.15.187:0/2739370506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c8074530 0x7f82c80ae310 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f82c80ba1e0 tx=0x7f82c001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b9d52b65ccfa702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:16.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82c5ffb640 1 -- 172.21.15.187:0/2739370506 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00a5640 con 0x7f82c8074530 2024-04-25T00:55:16.659 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82c5ffb640 1 -- 172.21.15.187:0/2739370506 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c00a57e0 con 0x7f82c8074530 2024-04-25T00:55:16.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82c5ffb640 1 -- 172.21.15.187:0/2739370506 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00a5bb0 con 0x7f82c8074530 2024-04-25T00:55:16.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 -- 172.21.15.187:0/2739370506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c8074530 msgr2=0x7f82c80ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 --2- 172.21.15.187:0/2739370506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c8074530 0x7f82c80ae310 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f82c80ba1e0 tx=0x7f82c001c890 comp rx=0 tx=0).stop 2024-04-25T00:55:16.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 -- 172.21.15.187:0/2739370506 shutdown_connections 2024-04-25T00:55:16.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 --2- 172.21.15.187:0/2739370506 >> v2:172.21.15.112:3300/0 conn(0x7f82c80ae850 0x7f82c814aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 --2- 172.21.15.187:0/2739370506 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c8074530 0x7f82c80ae310 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.660 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 -- 172.21.15.187:0/2739370506 >> 172.21.15.187:0/2739370506 conn(0x7f82c806b810 msgr2=0x7f82c80b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.661 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 -- 172.21.15.187:0/2739370506 shutdown_connections 2024-04-25T00:55:16.661 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 -- 172.21.15.187:0/2739370506 wait complete. 2024-04-25T00:55:16.661 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 Processor -- start 2024-04-25T00:55:16.661 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.653+0000 7f82cd674640 1 -- start start 2024-04-25T00:55:16.665 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82c8074530 0x7f82c81498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c80ae850 0x7f82c81e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c814af70 0x7f82c81412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f82c80b34b0 con 0x7f82c814af70 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f82c814b7d0 con 0x7f82c80ae850 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f82c80b3630 con 0x7f82c8074530 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c80ae850 0x7f82c81e02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c6ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82c8074530 0x7f82c81498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c80ae850 0x7f82c81e02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.187:60044/0 (socket says 172.21.15.187:60044) 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 -- 172.21.15.187:0/615233799 learned_addr learned my addr 172.21.15.187:0/615233799 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:16.666 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c77fe640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c814af70 0x7f82c81412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 -- 172.21.15.187:0/615233799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82c8074530 msgr2=0x7f82c81498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82c8074530 0x7f82c81498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 -- 172.21.15.187:0/615233799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c814af70 msgr2=0x7f82c81412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c814af70 0x7f82c81412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 -- 172.21.15.187:0/615233799 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c8071450 con 0x7f82c80ae850 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c6ffd640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82c8074530 0x7f82c81498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82c67fc640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c80ae850 0x7f82c81e02b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c7f0 tx=0x7f82bc00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82af7fe640 1 -- 172.21.15.187:0/615233799 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc015070 con 0x7f82c80ae850 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82af7fe640 1 -- 172.21.15.187:0/615233799 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc004240 con 0x7f82c80ae850 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c8072710 con 0x7f82c80ae850 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82af7fe640 1 -- 172.21.15.187:0/615233799 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc003620 con 0x7f82c80ae850 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c8072180 con 0x7f82c80ae850 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82af7fe640 1 -- 172.21.15.187:0/615233799 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f82bc0037c0 con 0x7f82c80ae850 2024-04-25T00:55:16.667 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82af7fe640 1 -- 172.21.15.187:0/615233799 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f82bc007df0 con 0x7f82c80ae850 2024-04-25T00:55:16.668 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.657+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c8074b60 con 0x7f82c80ae850 2024-04-25T00:55:16.668 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.661+0000 7f82af7fe640 1 -- 172.21.15.187:0/615233799 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f82bc019610 con 0x7f82c80ae850 2024-04-25T00:55:16.673 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.674+0000 7fd00f1a3180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:55:16.690 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:16.689+0000 7fb93e172180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:55:16.728 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.725+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "58cf3cc8-e445-4def-b50a-f502aa6a24c7", "id": 5} v 0) -- 0x7f82c80728e0 con 0x7f82c80ae850 2024-04-25T00:55:16.735 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82af7fe640 1 -- 172.21.15.187:0/615233799 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "58cf3cc8-e445-4def-b50a-f502aa6a24c7", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f82bc003c40 con 0x7f82c80ae850 2024-04-25T00:55:16.735 INFO:teuthology.orchestra.run.smithi187.stdout:5 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c80ae850 msgr2=0x7f82c81e02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c80ae850 0x7f82c81e02b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c7f0 tx=0x7f82bc00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 shutdown_connections 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82c814af70 0x7f82c81412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f82c80ae850 0x7f82c81e02b0 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c7f0 tx=0x7f82bc00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 --2- 172.21.15.187:0/615233799 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f82c8074530 0x7f82c81498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 >> 172.21.15.187:0/615233799 conn(0x7f82c806b810 msgr2=0x7f82c8147dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 shutdown_connections 2024-04-25T00:55:16.736 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.733+0000 7f82cd674640 1 -- 172.21.15.187:0/615233799 wait complete. 2024-04-25T00:55:16.751 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 35497ef1-a007-4874-a742-e7038034e1ad 6 2024-04-25T00:55:16.771 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:16.769+0000 7fb93e172180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:55:16.822 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.822+0000 7fd00f1a3180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:55:16.825 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:16.825+0000 7fb93e172180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:55:16.883 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:16.882+0000 7fd00f1a3180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:55:16.883 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f057e39b640 1 Processor -- start 2024-04-25T00:55:16.883 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f057e39b640 1 -- start start 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f057e39b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f05780b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f057e39b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f05781482f0 0x7f057814fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f057e39b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0578070ab0 con 0x7f05780b5f60 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f057e39b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0578148700 con 0x7f05781482f0 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f057e39b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0578070d20 con 0x7f05780671b0 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f0577fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f05780b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f0577fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f05780b5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:59214/0 (socket says 172.21.15.187:59214) 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f0577fff640 1 -- 172.21.15.187:0/3697153232 learned_addr learned my addr 172.21.15.187:0/3697153232 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.881+0000 7f05777fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f05780b5f60 0x7f0578147be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:56974/0 (socket says 172.21.15.187:56974) 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057c938640 1 --2- 172.21.15.187:0/3697153232 >> v2:172.21.15.112:3300/0 conn(0x7f05781482f0 0x7f057814fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.885 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0576ffd640 1 -- 172.21.15.187:0/3697153232 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1610273706 0 0) 0x7f0578070ab0 con 0x7f05780b5f60 2024-04-25T00:55:16.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0577fff640 1 -- 172.21.15.187:0/3697153232 >> v2:172.21.15.112:3300/0 conn(0x7f05781482f0 msgr2=0x7f057814fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:16.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0577fff640 1 --2- 172.21.15.187:0/3697153232 >> v2:172.21.15.112:3300/0 conn(0x7f05781482f0 0x7f057814fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0577fff640 1 -- 172.21.15.187:0/3697153232 >> v1:172.21.15.187:6789/0 conn(0x7f05780b5f60 legacy=0x7f0578147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.886 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0577fff640 1 -- 172.21.15.187:0/3697153232 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0578072b10 con 0x7f05780671b0 2024-04-25T00:55:16.887 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0577fff640 1 --2- 172.21.15.187:0/3697153232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f05780b5020 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f056c009870 tx=0x7f056c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e259f1e974cde2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:16.887 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0576ffd640 1 -- 172.21.15.187:0/3697153232 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c0b0070 con 0x7f05780671b0 2024-04-25T00:55:16.887 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0576ffd640 1 -- 172.21.15.187:0/3697153232 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c004230 con 0x7f05780671b0 2024-04-25T00:55:16.887 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f0576ffd640 1 -- 172.21.15.187:0/3697153232 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c003620 con 0x7f05780671b0 2024-04-25T00:55:16.888 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057e39b640 1 -- 172.21.15.187:0/3697153232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 msgr2=0x7f05780b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.888 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057e39b640 1 --2- 172.21.15.187:0/3697153232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f05780b5020 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f056c009870 tx=0x7f056c01c440 comp rx=0 tx=0).stop 2024-04-25T00:55:16.889 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057e39b640 1 -- 172.21.15.187:0/3697153232 shutdown_connections 2024-04-25T00:55:16.889 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057e39b640 1 --2- 172.21.15.187:0/3697153232 >> v2:172.21.15.112:3300/0 conn(0x7f05781482f0 0x7f057814fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.889 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057e39b640 1 --2- 172.21.15.187:0/3697153232 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f05780b5020 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.889 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057e39b640 1 -- 172.21.15.187:0/3697153232 >> 172.21.15.187:0/3697153232 conn(0x7f057806b3e0 msgr2=0x7f057806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.889 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057e39b640 1 -- 172.21.15.187:0/3697153232 shutdown_connections 2024-04-25T00:55:16.889 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.885+0000 7f057e39b640 1 -- 172.21.15.187:0/3697153232 wait complete. 2024-04-25T00:55:16.891 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 Processor -- start 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 -- start start 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f057809b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05780b5f60 0x7f057809d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05781482f0 0x7f05780b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f05780b5560 con 0x7f05780b5f60 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f057806e610 con 0x7f05781482f0 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f057809e500 con 0x7f05780671b0 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f057809b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f057809b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:59218/0 (socket says 172.21.15.187:59218) 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 -- 172.21.15.187:0/3342074421 learned_addr learned my addr 172.21.15.187:0/3342074421 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057c938640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05781482f0 0x7f05780b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f05777fe640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05780b5f60 0x7f057809d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:16.892 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 -- 172.21.15.187:0/3342074421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05781482f0 msgr2=0x7f05780b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05781482f0 0x7f05780b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 -- 172.21.15.187:0/3342074421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05780b5f60 msgr2=0x7f057809d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05780b5f60 0x7f057809d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 -- 172.21.15.187:0/3342074421 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057809cac0 con 0x7f05780671b0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0577fff640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f057809b8c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f056c01a070 tx=0x7f056c0191e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0574ff9640 1 -- 172.21.15.187:0/3342074421 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c0bb070 con 0x7f05780671b0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0574ff9640 1 -- 172.21.15.187:0/3342074421 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f056c0b6640 con 0x7f05780671b0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0574ff9640 1 -- 172.21.15.187:0/3342074421 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056c0b0040 con 0x7f05780671b0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057809cda0 con 0x7f05780671b0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f057809c7e0 con 0x7f05780671b0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0574ff9640 1 -- 172.21.15.187:0/3342074421 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f056c0040f0 con 0x7f05780671b0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f0574ff9640 1 -- 172.21.15.187:0/3342074421 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f056c0abc60 con 0x7f05780671b0 2024-04-25T00:55:16.893 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.889+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f057809e290 con 0x7f05780671b0 2024-04-25T00:55:16.894 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.893+0000 7f0574ff9640 1 -- 172.21.15.187:0/3342074421 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f057809e290 con 0x7f05780671b0 2024-04-25T00:55:16.964 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.961+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd new", "uuid": "35497ef1-a007-4874-a742-e7038034e1ad", "id": 6} v 0) -- 0x7f057804f510 con 0x7f05780671b0 2024-04-25T00:55:16.969 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f0574ff9640 1 -- 172.21.15.187:0/3342074421 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "35497ef1-a007-4874-a742-e7038034e1ad", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f056c016150 con 0x7f05780671b0 2024-04-25T00:55:16.969 INFO:teuthology.orchestra.run.smithi187.stdout:6 2024-04-25T00:55:16.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 msgr2=0x7f057809b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:16.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f057809b8c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f056c01a070 tx=0x7f056c0191e0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 shutdown_connections 2024-04-25T00:55:16.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f05781482f0 0x7f05780b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05780b5f60 0x7f057809d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.971 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 --2- 172.21.15.187:0/3342074421 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f05780671b0 0x7f057809b8c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:16.972 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 >> 172.21.15.187:0/3342074421 conn(0x7f057806b3e0 msgr2=0x7f05780b6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:16.972 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 shutdown_connections 2024-04-25T00:55:16.972 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:16.969+0000 7f057e39b640 1 -- 172.21.15.187:0/3342074421 wait complete. 2024-04-25T00:55:16.982 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:16.981+0000 7fb93e172180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:55:16.986 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 507622e0-4d62-4865-95a0-1c25f5c3a209 7 2024-04-25T00:55:17.047 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:17.045+0000 7fb93e172180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:55:17.101 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c78312640 1 Processor -- start 2024-04-25T00:55:17.101 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c78312640 1 -- start start 2024-04-25T00:55:17.102 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c78312640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c700723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c78312640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c700b6c90 con 0x7f4c7014fd80 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c78312640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c700b5fb0 con 0x7f4c700b68b0 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c78312640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7013e300 con 0x7f4c7014aa00 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c760ae640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f4c700b68b0 0x7f4c7013dbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.187:56982/0 (socket says 172.21.15.187:56982) 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c758ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c700723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c760ae640 1 -- 172.21.15.187:0/1061677269 learned_addr learned my addr 172.21.15.187:0/1061677269 (peer_addr_for_me v1:172.21.15.187:0/0) 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c750ac640 1 -- 172.21.15.187:0/1061677269 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 481382488 0 0) 0x7f4c700b5fb0 con 0x7f4c700b68b0 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c750ac640 1 -- 172.21.15.187:0/1061677269 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4c5c003530 con 0x7f4c700b68b0 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c758ad640 1 -- 172.21.15.187:0/1061677269 >> v1:172.21.15.187:6789/0 conn(0x7f4c700b68b0 legacy=0x7f4c7013dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.103 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c758ad640 1 -- 172.21.15.187:0/1061677269 >> v1:172.21.15.112:6789/0 conn(0x7f4c7014fd80 legacy=0x7f4c70152170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.104 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.101+0000 7f4c758ad640 1 -- 172.21.15.187:0/1061677269 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c70071eb0 con 0x7f4c7014aa00 2024-04-25T00:55:17.106 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c758ad640 1 --2- 172.21.15.187:0/1061677269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c700723d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009820 tx=0x7f4c6001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e4b5ca6f232f00f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:17.106 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c750ac640 1 -- 172.21.15.187:0/1061677269 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600b1070 con 0x7f4c7014aa00 2024-04-25T00:55:17.106 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c750ac640 1 -- 172.21.15.187:0/1061677269 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c600ac630 con 0x7f4c7014aa00 2024-04-25T00:55:17.106 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c750ac640 1 -- 172.21.15.187:0/1061677269 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600a8050 con 0x7f4c7014aa00 2024-04-25T00:55:17.107 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 -- 172.21.15.187:0/1061677269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 msgr2=0x7f4c700723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.107 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 --2- 172.21.15.187:0/1061677269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c700723d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009820 tx=0x7f4c6001c4c0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.107 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 -- 172.21.15.187:0/1061677269 shutdown_connections 2024-04-25T00:55:17.107 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 --2- 172.21.15.187:0/1061677269 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c700723d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.107 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 -- 172.21.15.187:0/1061677269 >> 172.21.15.187:0/1061677269 conn(0x7f4c7006d110 msgr2=0x7f4c7006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:17.107 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 -- 172.21.15.187:0/1061677269 shutdown_connections 2024-04-25T00:55:17.107 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 -- 172.21.15.187:0/1061677269 wait complete. 2024-04-25T00:55:17.108 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 Processor -- start 2024-04-25T00:55:17.108 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.105+0000 7f4c78312640 1 -- start start 2024-04-25T00:55:17.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c700b68b0 0x7f4c7013fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c701e4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7014fd80 0x7f4c700b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70141b50 con 0x7f4c7014fd80 2024-04-25T00:55:17.110 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4c701419d0 con 0x7f4c700b68b0 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4c701564c0 con 0x7f4c7014aa00 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c758ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c701e4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c760ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c700b68b0 0x7f4c7013fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c758ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c701e4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.187:59232/0 (socket says 172.21.15.187:59232) 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c758ad640 1 -- 172.21.15.187:0/759854332 learned_addr learned my addr 172.21.15.187:0/759854332 (peer_addr_for_me v2:172.21.15.187:0/0) 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c760ae640 1 -- 172.21.15.187:0/759854332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 msgr2=0x7f4c701e4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c768af640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7014fd80 0x7f4c700b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c760ae640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c701e4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c760ae640 1 -- 172.21.15.187:0/759854332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7014fd80 msgr2=0x7f4c700b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.111 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c760ae640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7014fd80 0x7f4c700b2dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c760ae640 1 -- 172.21.15.187:0/759854332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7013f8b0 con 0x7f4c700b68b0 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c758ad640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c701e4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c768af640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7014fd80 0x7f4c700b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c760ae640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c700b68b0 0x7f4c7013fd40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00c400 tx=0x7f4c6c00c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c66ffd640 1 -- 172.21.15.187:0/759854332 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c017070 con 0x7f4c700b68b0 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c66ffd640 1 -- 172.21.15.187:0/759854332 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6c005cf0 con 0x7f4c700b68b0 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7013f5d0 con 0x7f4c700b68b0 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c66ffd640 1 -- 172.21.15.187:0/759854332 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0072d0 con 0x7f4c700b68b0 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c701e3910 con 0x7f4c700b68b0 2024-04-25T00:55:17.112 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c701e3c40 con 0x7f4c700b68b0 2024-04-25T00:55:17.113 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c66ffd640 1 -- 172.21.15.187:0/759854332 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4c6c01a070 con 0x7f4c700b68b0 2024-04-25T00:55:17.113 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.109+0000 7f4c66ffd640 1 -- 172.21.15.187:0/759854332 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f4c6c01cc60 con 0x7f4c700b68b0 2024-04-25T00:55:17.113 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.113+0000 7f4c66ffd640 1 -- 172.21.15.187:0/759854332 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4c7013f8b0 con 0x7f4c700b68b0 2024-04-25T00:55:17.180 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.177+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_command({"prefix": "osd new", "uuid": "507622e0-4d62-4865-95a0-1c25f5c3a209", "id": 7} v 0) -- 0x7f4c7004f5f0 con 0x7f4c700b68b0 2024-04-25T00:55:17.190 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c66ffd640 1 -- 172.21.15.187:0/759854332 <== mon.1 v2:172.21.15.187:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "507622e0-4d62-4865-95a0-1c25f5c3a209", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f4c6c007470 con 0x7f4c700b68b0 2024-04-25T00:55:17.190 INFO:teuthology.orchestra.run.smithi187.stdout:7 2024-04-25T00:55:17.191 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c700b68b0 msgr2=0x7f4c7013fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.191 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c700b68b0 0x7f4c7013fd40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00c400 tx=0x7f4c6c00c8c0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.191 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 shutdown_connections 2024-04-25T00:55:17.192 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4c7014fd80 0x7f4c700b2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.192 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4c7014aa00 0x7f4c701e4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.192 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 --2- 172.21.15.187:0/759854332 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4c700b68b0 0x7f4c7013fd40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.192 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 >> 172.21.15.187:0/759854332 conn(0x7f4c7006d110 msgr2=0x7f4c70146200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:17.192 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 shutdown_connections 2024-04-25T00:55:17.192 INFO:teuthology.orchestra.run.smithi187.stderr:2024-04-25T00:55:17.189+0000 7f4c78312640 1 -- 172.21.15.187:0/759854332 wait complete. 2024-04-25T00:55:17.206 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T00:55:17.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T00:55:17.209 INFO:tasks.ceph.osd.0:Started 2024-04-25T00:55:17.209 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T00:55:17.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T00:55:17.211 INFO:tasks.ceph.osd.1:Started 2024-04-25T00:55:17.212 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T00:55:17.212 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T00:55:17.214 INFO:tasks.ceph.osd.2:Started 2024-04-25T00:55:17.214 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T00:55:17.214 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T00:55:17.216 INFO:tasks.ceph.osd.3:Started 2024-04-25T00:55:17.217 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T00:55:17.217 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T00:55:17.219 INFO:tasks.ceph.osd.4:Started 2024-04-25T00:55:17.219 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T00:55:17.219 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T00:55:17.221 INFO:tasks.ceph.osd.5:Started 2024-04-25T00:55:17.221 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T00:55:17.221 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T00:55:17.223 INFO:tasks.ceph.osd.6:Started 2024-04-25T00:55:17.223 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T00:55:17.223 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T00:55:17.225 INFO:tasks.ceph.osd.7:Started 2024-04-25T00:55:17.225 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:17.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.374+0000 7fc86375c640 1 Processor -- start 2024-04-25T00:55:17.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.374+0000 7fc86375c640 1 -- start start 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc85c09c270 0x7fc85c147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc85c147650 0x7fc85c148240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc85c0bbec0 con 0x7fc85c0722d0 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc85c0baaf0 con 0x7fc85c09c270 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc85c0bc5c0 con 0x7fc85c147650 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc860cf7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc85c09c270 0x7fc85c147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc860cf7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc85c09c270 0x7fc85c147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59372/0 (socket says 172.21.15.112:59372) 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc860cf7640 1 -- 172.21.15.112:0/115352658 learned_addr learned my addr 172.21.15.112:0/115352658 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc861cf9640 1 --2- 172.21.15.112:0/115352658 >> v2:172.21.15.187:3300/0 conn(0x7fc85c147650 0x7fc85c148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc860cf7640 1 -- 172.21.15.112:0/115352658 >> v2:172.21.15.187:3300/0 conn(0x7fc85c147650 msgr2=0x7fc85c148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc860cf7640 1 --2- 172.21.15.112:0/115352658 >> v2:172.21.15.187:3300/0 conn(0x7fc85c147650 0x7fc85c148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc860cf7640 1 -- 172.21.15.112:0/115352658 >> v1:172.21.15.187:6789/0 conn(0x7fc85c0722d0 legacy=0x7fc85c072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc860cf7640 1 -- 172.21.15.112:0/115352658 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c0bc8a0 con 0x7fc85c09c270 2024-04-25T00:55:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc860cf7640 1 --2- 172.21.15.112:0/115352658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c09c270 0x7fc85c147d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc85c14bb80 tx=0x7fc84c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e9aba1b4619409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc853fff640 1 -- 172.21.15.112:0/115352658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0aa650 con 0x7fc85c09c270 2024-04-25T00:55:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc853fff640 1 -- 172.21.15.112:0/115352658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84c0af070 con 0x7fc85c09c270 2024-04-25T00:55:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc853fff640 1 -- 172.21.15.112:0/115352658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0aaa10 con 0x7fc85c09c270 2024-04-25T00:55:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 -- 172.21.15.112:0/115352658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c09c270 msgr2=0x7fc85c147d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 --2- 172.21.15.112:0/115352658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c09c270 0x7fc85c147d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc85c14bb80 tx=0x7fc84c01c4f0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 -- 172.21.15.112:0/115352658 shutdown_connections 2024-04-25T00:55:17.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 --2- 172.21.15.112:0/115352658 >> v2:172.21.15.187:3300/0 conn(0x7fc85c147650 0x7fc85c148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 --2- 172.21.15.112:0/115352658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c09c270 0x7fc85c147d00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 -- 172.21.15.112:0/115352658 >> 172.21.15.112:0/115352658 conn(0x7fc85c09e8c0 msgr2=0x7fc85c09ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:17.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 -- 172.21.15.112:0/115352658 shutdown_connections 2024-04-25T00:55:17.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.378+0000 7fc86375c640 1 -- 172.21.15.112:0/115352658 wait complete. 2024-04-25T00:55:17.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 Processor -- start 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 -- start start 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c0722d0 0x7fc85c148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85c09c270 0x7fc85c140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85c147650 0x7fc85c1e3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c154890 con 0x7fc85c0722d0 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c154710 con 0x7fc85c147650 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc85c154a10 con 0x7fc85c09c270 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c0722d0 0x7fc85c148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c0722d0 0x7fc85c148c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59384/0 (socket says 172.21.15.112:59384) 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 -- 172.21.15.112:0/4108889690 learned_addr learned my addr 172.21.15.112:0/4108889690 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc860cf7640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85c09c270 0x7fc85c140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.384 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc861cf9640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85c147650 0x7fc85c1e3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 -- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85c09c270 msgr2=0x7fc85c140990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85c09c270 0x7fc85c140990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 -- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85c147650 msgr2=0x7fc85c1e3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85c147650 0x7fc85c1e3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 -- 172.21.15.112:0/4108889690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c1dfed0 con 0x7fc85c0722d0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc861cf9640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85c147650 0x7fc85c1e3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc8614f8640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c0722d0 0x7fc85c148c70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c3c0 tx=0x7fc85800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc851ffb640 1 -- 172.21.15.112:0/4108889690 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858017070 con 0x7fc85c0722d0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc851ffb640 1 -- 172.21.15.112:0/4108889690 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8580052d0 con 0x7fc85c0722d0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c14b7a0 con 0x7fc85c0722d0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc851ffb640 1 -- 172.21.15.112:0/4108889690 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858003840 con 0x7fc85c0722d0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c1e3610 con 0x7fc85c0722d0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc851ffb640 1 -- 172.21.15.112:0/4108889690 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc85801a070 con 0x7fc85c0722d0 2024-04-25T00:55:17.385 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc851ffb640 1 -- 172.21.15.112:0/4108889690 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc858011b80 con 0x7fc85c0722d0 2024-04-25T00:55:17.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.382+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85c04f580 con 0x7fc85c0722d0 2024-04-25T00:55:17.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.394+0000 7fc851ffb640 1 -- 172.21.15.112:0/4108889690 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc858021080 con 0x7fc85c0722d0 2024-04-25T00:55:17.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc85c14b4c0 con 0x7fc85c0722d0 2024-04-25T00:55:17.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc851ffb640 1 -- 172.21.15.112:0/4108889690 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc858011590 con 0x7fc85c0722d0 2024-04-25T00:55:17.467 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:17.467 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":10,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:17.184543+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c0722d0 msgr2=0x7fc85c148c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c0722d0 0x7fc85c148c70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c3c0 tx=0x7fc85800c880 comp rx=0 tx=0).stop 2024-04-25T00:55:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 shutdown_connections 2024-04-25T00:55:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc85c147650 0x7fc85c1e3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc85c09c270 0x7fc85c140990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 --2- 172.21.15.112:0/4108889690 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc85c0722d0 0x7fc85c148c70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 >> 172.21.15.112:0/4108889690 conn(0x7fc85c09e8c0 msgr2=0x7fc85c0b7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 shutdown_connections 2024-04-25T00:55:17.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:17.466+0000 7fc86375c640 1 -- 172.21.15.112:0/4108889690 wait complete. 2024-04-25T00:55:17.483 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T00:55:17.483 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T00:55:17.555 INFO:tasks.ceph.mgr.y.smithi112.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:55:17.555 INFO:tasks.ceph.mgr.y.smithi112.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:55:17.555 INFO:tasks.ceph.mgr.y.smithi112.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:55:17.608 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:17.606+0000 7fd00f1a3180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:55:17.733 INFO:tasks.ceph.mgr.x.smithi187.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T00:55:17.733 INFO:tasks.ceph.mgr.x.smithi187.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T00:55:17.733 INFO:tasks.ceph.mgr.x.smithi187.stderr: from numpy import show_config as show_numpy_config 2024-04-25T00:55:17.779 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:17.778+0000 7fd00f1a3180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:55:17.793 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:17.793+0000 7fb93e172180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:55:17.815 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T00:55:17.813+0000 7f1ec1b6e7c0 -1 Falling back to public interface 2024-04-25T00:55:17.815 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T00:55:17.813+0000 7fdf0714d7c0 -1 Falling back to public interface 2024-04-25T00:55:17.846 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T00:55:17.845+0000 7f2fe715f7c0 -1 Falling back to public interface 2024-04-25T00:55:17.849 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T00:55:17.849+0000 7ff03d2667c0 -1 Falling back to public interface 2024-04-25T00:55:17.862 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-04-25T00:55:17.862+0000 7fc921dbf7c0 -1 Falling back to public interface 2024-04-25T00:55:17.863 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-04-25T00:55:17.862+0000 7f2a316f37c0 -1 Falling back to public interface 2024-04-25T00:55:17.881 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-04-25T00:55:17.882+0000 7fc170f577c0 -1 Falling back to public interface 2024-04-25T00:55:17.882 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-04-25T00:55:17.882+0000 7f6c8f5047c0 -1 Falling back to public interface 2024-04-25T00:55:17.947 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:17.946+0000 7fd00f1a3180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:55:17.979 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:17.977+0000 7fb93e172180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:55:18.016 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:18.014+0000 7fd00f1a3180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:55:18.153 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:18.153+0000 7fb93e172180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:55:18.163 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:18.162+0000 7fd00f1a3180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:55:18.224 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:18.221+0000 7fb93e172180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:55:18.288 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:18.371 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:18.369+0000 7fb93e172180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:55:18.392 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.390+0000 7f374c57f640 1 Processor -- start 2024-04-25T00:55:18.393 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.390+0000 7f374c57f640 1 -- start start 2024-04-25T00:55:18.395 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f374c57f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f37440743f0 0x7f374414b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:18.395 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f374c57f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37440b76f0 con 0x7f37440b9fe0 2024-04-25T00:55:18.395 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f374c57f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37440747d0 con 0x7f37440b4640 2024-04-25T00:55:18.395 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f374c57f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f37440b8a10 con 0x7f37440743f0 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f37440b9fe0 0x7f37440b3f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41446/0 (socket says 172.21.15.112:41446) 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 -- 172.21.15.112:0/1828949063 learned_addr learned my addr 172.21.15.112:0/1828949063 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f374a31b640 1 --2- 172.21.15.112:0/1828949063 >> v2:172.21.15.187:3300/0 conn(0x7f37440743f0 0x7f374414b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1504800995 0 0) 0x7f37440b76f0 con 0x7f37440b9fe0 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3730003530 con 0x7f37440b9fe0 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3533753042 0 0) 0x7f3730003530 con 0x7f37440b9fe0 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 >> v2:172.21.15.187:3300/0 conn(0x7f37440743f0 msgr2=0x7f374414b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 --2- 172.21.15.112:0/1828949063 >> v2:172.21.15.187:3300/0 conn(0x7f37440743f0 0x7f374414b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 >> v1:172.21.15.187:6789/0 conn(0x7f37440b4640 legacy=0x7f3744151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3744098cb0 con 0x7f37440b9fe0 2024-04-25T00:55:18.396 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f3734001680 con 0x7f37440b9fe0 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 >> v1:172.21.15.112:6789/0 conn(0x7f37440b9fe0 legacy=0x7f37440b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37300039c0 0x7f3730003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730004810 0x7f3730004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37300052f0 0x7f373000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37440b8a10 con 0x7f3730004810 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f373000c1c0 con 0x7f37300039c0 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f373000c3c0 con 0x7f37300052f0 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730004810 0x7f3730004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 -- 172.21.15.112:0/1828949063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37300052f0 msgr2=0x7f373000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37300052f0 0x7f373000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.397 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 -- 172.21.15.112:0/1828949063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37300039c0 msgr2=0x7f3730003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:18.398 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37300039c0 0x7f3730003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.398 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 -- 172.21.15.112:0/1828949063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3730003530 con 0x7f3730004810 2024-04-25T00:55:18.398 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f374ab1c640 1 -- 172.21.15.112:0/1828949063 reap_dead start 2024-04-25T00:55:18.402 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749b1a640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730004810 0x7f3730004c90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3744067f60 tx=0x7f373401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3d61021062c455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:18.402 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734007d20 con 0x7f3730004810 2024-04-25T00:55:18.402 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37340a9070 con 0x7f3730004810 2024-04-25T00:55:18.402 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.394+0000 7f3749319640 1 -- 172.21.15.112:0/1828949063 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734005da0 con 0x7f3730004810 2024-04-25T00:55:18.402 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- 172.21.15.112:0/1828949063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730004810 msgr2=0x7f3730004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:18.402 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730004810 0x7f3730004c90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3744067f60 tx=0x7f373401cbe0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.403 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- 172.21.15.112:0/1828949063 shutdown_connections 2024-04-25T00:55:18.403 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 --2- 172.21.15.112:0/1828949063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3730004810 0x7f3730004c90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.403 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- 172.21.15.112:0/1828949063 >> 172.21.15.112:0/1828949063 conn(0x7f37440b9430 msgr2=0x7f37440b9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:18.403 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- 172.21.15.112:0/1828949063 shutdown_connections 2024-04-25T00:55:18.403 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- 172.21.15.112:0/1828949063 wait complete. 2024-04-25T00:55:18.403 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 Processor -- start 2024-04-25T00:55:18.404 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- start start 2024-04-25T00:55:18.404 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3730004810 0x7f37441469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:18.404 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3744072240 0x7f37441e4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:18.404 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f374409c2d0 0x7f374409c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:18.404 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37440b5ec0 con 0x7f3744072240 2024-04-25T00:55:18.404 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f37440b87f0 con 0x7f3730004810 2024-04-25T00:55:18.404 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f37440b6040 con 0x7f374409c2d0 2024-04-25T00:55:18.405 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3744072240 0x7f37441e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:18.405 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3744072240 0x7f37441e4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:59396/0 (socket says 172.21.15.112:59396) 2024-04-25T00:55:18.405 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 -- 172.21.15.112:0/1534936658 learned_addr learned my addr 172.21.15.112:0/1534936658 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:18.405 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374a31b640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3730004810 0x7f37441469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:18.405 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374ab1c640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f374409c2d0 0x7f374409c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:18.405 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 -- 172.21.15.112:0/1534936658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f374409c2d0 msgr2=0x7f374409c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:18.405 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f374409c2d0 0x7f374409c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.406 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 -- 172.21.15.112:0/1534936658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3730004810 msgr2=0x7f37441469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:18.406 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3730004810 0x7f37441469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.406 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 -- 172.21.15.112:0/1534936658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374409d3b0 con 0x7f3744072240 2024-04-25T00:55:18.406 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f3749b1a640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3744072240 0x7f37441e4400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3734001980 tx=0x7f3730005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:18.406 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f373affd640 1 -- 172.21.15.112:0/1534936658 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734007d20 con 0x7f3744072240 2024-04-25T00:55:18.406 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f373affd640 1 -- 172.21.15.112:0/1534936658 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3734004e50 con 0x7f3744072240 2024-04-25T00:55:18.406 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3744072a50 con 0x7f3744072240 2024-04-25T00:55:18.406 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f373affd640 1 -- 172.21.15.112:0/1534936658 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3734001480 con 0x7f3744072240 2024-04-25T00:55:18.407 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374409e380 con 0x7f3744072240 2024-04-25T00:55:18.407 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f373affd640 1 -- 172.21.15.112:0/1534936658 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f374414ba30 con 0x7f3744072240 2024-04-25T00:55:18.407 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f373affd640 1 -- 172.21.15.112:0/1534936658 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f37340aeb30 con 0x7f3744072240 2024-04-25T00:55:18.407 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.398+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37441e4030 con 0x7f3744072240 2024-04-25T00:55:18.408 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.402+0000 7f373affd640 1 -- 172.21.15.112:0/1534936658 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f37340168f0 con 0x7f3744072240 2024-04-25T00:55:18.459 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f374406d250 con 0x7f3744072240 2024-04-25T00:55:18.460 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f373affd640 1 -- 172.21.15.112:0/1534936658 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f3734001620 con 0x7f3744072240 2024-04-25T00:55:18.460 INFO:teuthology.misc.health.smithi112.stdout: 2024-04-25T00:55:18.461 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":10,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:17.184543+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:18.462 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3744072240 msgr2=0x7f37441e4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:18.462 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3744072240 0x7f37441e4400 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3734001980 tx=0x7f3730005840 comp rx=0 tx=0).stop 2024-04-25T00:55:18.463 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 shutdown_connections 2024-04-25T00:55:18.463 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f374409c2d0 0x7f374409c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.463 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3744072240 0x7f37441e4400 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.463 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 --2- 172.21.15.112:0/1534936658 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3730004810 0x7f37441469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:18.463 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 >> 172.21.15.112:0/1534936658 conn(0x7f37440b9430 msgr2=0x7f37440b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:18.464 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 shutdown_connections 2024-04-25T00:55:18.464 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:18.458+0000 7f374c57f640 1 -- 172.21.15.112:0/1534936658 wait complete. 2024-04-25T00:55:18.474 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T00:55:18.798 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:18.798+0000 7fd00f1a3180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:55:18.874 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:18.874+0000 7fd00f1a3180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:55:19.010 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:19.009+0000 7fb93e172180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:55:19.054 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:19.054+0000 7fd00f1a3180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:55:19.090 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:19.085+0000 7fb93e172180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:55:19.127 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:19.126+0000 7fd00f1a3180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:55:19.267 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:19.265+0000 7fb93e172180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:55:19.340 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:19.337+0000 7fb93e172180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:55:19.417 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:19.414+0000 7fd00f1a3180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:55:19.478 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:19.478+0000 7fd00f1a3180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:55:19.637 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:19.637+0000 7fb93e172180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:55:19.699 INFO:tasks.ceph.mgr.x.smithi187.stderr:2024-04-25T00:55:19.697+0000 7fb93e172180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:55:20.524 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:20.521+0000 7fcfb64e9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T00:55:20.524 INFO:tasks.ceph.mgr.y.smithi112.stderr:2024-04-25T00:55:20.521+0000 7fcfb64e9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T00:55:21.639 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T00:55:21.637+0000 7f2fe715f7c0 -1 osd.5 0 log_to_monitors true 2024-04-25T00:55:21.722 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-04-25T00:55:21.721+0000 7fc921dbf7c0 -1 osd.3 0 log_to_monitors true 2024-04-25T00:55:21.758 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T00:55:21.757+0000 7fdf0714d7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T00:55:21.767 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T00:55:21.765+0000 7ff03d2667c0 -1 osd.7 0 log_to_monitors true 2024-04-25T00:55:21.806 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T00:55:21.805+0000 7f1ec1b6e7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T00:55:21.830 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-04-25T00:55:21.829+0000 7f6c8f5047c0 -1 osd.2 0 log_to_monitors true 2024-04-25T00:55:21.849 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-04-25T00:55:21.849+0000 7fc170f577c0 -1 osd.1 0 log_to_monitors true 2024-04-25T00:55:21.864 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-04-25T00:55:21.861+0000 7f2a316f37c0 -1 osd.0 0 log_to_monitors true 2024-04-25T00:55:25.281 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:25.445 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4b00392640 1 Processor -- start 2024-04-25T00:55:25.446 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4b00392640 1 -- start start 2024-04-25T00:55:25.446 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4b00392640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4af806bb80 0x7f4af80b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:25.446 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4b00392640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af8072df0 0x7f4af80b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:25.446 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4b00392640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4af80b0f30 con 0x7f4af80b3090 2024-04-25T00:55:25.446 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4b00392640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4af81495d0 con 0x7f4af806bb80 2024-04-25T00:55:25.447 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4b00392640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4af806bf60 con 0x7f4af8072df0 2024-04-25T00:55:25.447 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afe12e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4af806bb80 0x7f4af80b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afd92d640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f4af80b3090 0x7f4af80726e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:46740/0 (socket says 172.21.15.112:46740) 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afd92d640 1 -- 172.21.15.112:0/2936424764 learned_addr learned my addr 172.21.15.112:0/2936424764 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afe92f640 1 --2- 172.21.15.112:0/2936424764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af8072df0 0x7f4af80b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afe92f640 1 -- 172.21.15.112:0/2936424764 >> v2:172.21.15.112:3300/0 conn(0x7f4af806bb80 msgr2=0x7f4af80b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afd12c640 1 -- 172.21.15.112:0/2936424764 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87414927 0 0) 0x7f4af80b0f30 con 0x7f4af80b3090 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afe92f640 1 --2- 172.21.15.112:0/2936424764 >> v2:172.21.15.112:3300/0 conn(0x7f4af806bb80 0x7f4af80b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afe92f640 1 -- 172.21.15.112:0/2936424764 >> v1:172.21.15.187:6789/0 conn(0x7f4af80b3090 legacy=0x7f4af80726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afe92f640 1 -- 172.21.15.112:0/2936424764 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af80b56f0 con 0x7f4af8072df0 2024-04-25T00:55:25.448 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.445+0000 7f4afe12e640 1 --2- 172.21.15.112:0/2936424764 >> v2:172.21.15.112:3300/0 conn(0x7f4af806bb80 0x7f4af80b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:25.451 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4afe92f640 1 --2- 172.21.15.112:0/2936424764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af8072df0 0x7f4af80b0330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009d90 tx=0x7f4af401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141a6c58d5153300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:25.452 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4afd12c640 1 -- 172.21.15.112:0/2936424764 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af40b2070 con 0x7f4af8072df0 2024-04-25T00:55:25.452 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4afd12c640 1 -- 172.21.15.112:0/2936424764 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4af40a59e0 con 0x7f4af8072df0 2024-04-25T00:55:25.453 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4b00392640 1 -- 172.21.15.112:0/2936424764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af8072df0 msgr2=0x7f4af80b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:25.453 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4b00392640 1 --2- 172.21.15.112:0/2936424764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af8072df0 0x7f4af80b0330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009d90 tx=0x7f4af401c940 comp rx=0 tx=0).stop 2024-04-25T00:55:25.453 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4b00392640 1 -- 172.21.15.112:0/2936424764 shutdown_connections 2024-04-25T00:55:25.453 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4b00392640 1 --2- 172.21.15.112:0/2936424764 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af8072df0 0x7f4af80b0330 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.453 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4b00392640 1 --2- 172.21.15.112:0/2936424764 >> v2:172.21.15.112:3300/0 conn(0x7f4af806bb80 0x7f4af80b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.453 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4b00392640 1 -- 172.21.15.112:0/2936424764 >> 172.21.15.112:0/2936424764 conn(0x7f4af80b9430 msgr2=0x7f4af806b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:25.453 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.449+0000 7f4b00392640 1 -- 172.21.15.112:0/2936424764 shutdown_connections 2024-04-25T00:55:25.453 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 -- 172.21.15.112:0/2936424764 wait complete. 2024-04-25T00:55:25.455 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 Processor -- start 2024-04-25T00:55:25.455 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 -- start start 2024-04-25T00:55:25.456 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4af806bb80 0x7f4af8145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:25.456 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af8072df0 0x7f4af81e7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:25.456 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af80b3090 0x7f4af8140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:25.456 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8158d60 con 0x7f4af8072df0 2024-04-25T00:55:25.456 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4af8158be0 con 0x7f4af806bb80 2024-04-25T00:55:25.457 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4b00392640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4af8158ee0 con 0x7f4af80b3090 2024-04-25T00:55:25.457 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af8072df0 0x7f4af81e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:25.457 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af8072df0 0x7f4af81e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33484/0 (socket says 172.21.15.112:33484) 2024-04-25T00:55:25.457 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 -- 172.21.15.112:0/2992416278 learned_addr learned my addr 172.21.15.112:0/2992416278 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:25.457 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afe12e640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4af806bb80 0x7f4af8145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:25.457 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afe92f640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af80b3090 0x7f4af8140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 -- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af80b3090 msgr2=0x7f4af8140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af80b3090 0x7f4af8140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 -- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4af806bb80 msgr2=0x7f4af8145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4af806bb80 0x7f4af8145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 -- 172.21.15.112:0/2992416278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af8072050 con 0x7f4af8072df0 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afe12e640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4af806bb80 0x7f4af8145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.453+0000 7f4afd92d640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af8072df0 0x7f4af81e7880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c440 tx=0x7f4ae800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4aeeffd640 1 -- 172.21.15.112:0/2992416278 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8005cf0 con 0x7f4af8072df0 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4aeeffd640 1 -- 172.21.15.112:0/2992416278 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae8007070 con 0x7f4af8072df0 2024-04-25T00:55:25.458 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4aeeffd640 1 -- 172.21.15.112:0/2992416278 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8007460 con 0x7f4af8072df0 2024-04-25T00:55:25.459 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af806f6d0 con 0x7f4af8072df0 2024-04-25T00:55:25.459 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af8141cf0 con 0x7f4af8072df0 2024-04-25T00:55:25.459 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af8068b90 con 0x7f4af8072df0 2024-04-25T00:55:25.459 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4aeeffd640 1 -- 172.21.15.112:0/2992416278 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f4ae8007600 con 0x7f4af8072df0 2024-04-25T00:55:25.460 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4aeeffd640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4adc067830 0x7f4adc069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:25.460 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4afe12e640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4adc067830 0x7f4adc069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:25.460 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4aeeffd640 1 -- 172.21.15.112:0/2992416278 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f4ae8084910 con 0x7f4af8072df0 2024-04-25T00:55:25.461 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.457+0000 7f4afe12e640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4adc067830 0x7f4adc069cf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4af8145f10 tx=0x7f4af0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:25.475 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.473+0000 7f4aeeffd640 1 -- 172.21.15.112:0/2992416278 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4ae80562d0 con 0x7f4af8072df0 2024-04-25T00:55:25.605 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T00:55:25.605+0000 7ff03a222640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T00:55:25.610 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T00:55:25.609+0000 7f1ebd2f4640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T00:55:25.621 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-04-25T00:55:25.621+0000 7f6c8ac8a640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T00:55:25.629 INFO:tasks.ceph.osd.7.smithi187.stderr:2024-04-25T00:55:25.629+0000 7ff031033640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:55:25.647 INFO:tasks.ceph.osd.2.smithi112.stderr:2024-04-25T00:55:25.645+0000 7f6c832d1640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:55:25.649 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T00:55:25.649+0000 7f2fe28e5640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T00:55:25.653 INFO:tasks.ceph.osd.4.smithi187.stderr:2024-04-25T00:55:25.653+0000 7f1eb593b640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:55:25.681 INFO:tasks.ceph.osd.5.smithi187.stderr:2024-04-25T00:55:25.681+0000 7f2fdaf2c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:55:25.698 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T00:55:25.697+0000 7fdf028d3640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T00:55:25.726 INFO:tasks.ceph.osd.6.smithi187.stderr:2024-04-25T00:55:25.725+0000 7fdefaf1a640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:55:25.764 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-04-25T00:55:25.761+0000 7fc91d545640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T00:55:25.791 INFO:tasks.ceph.osd.3.smithi112.stderr:2024-04-25T00:55:25.789+0000 7fc915b8c640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:55:25.792 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.789+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4af814acf0 con 0x7f4af8072df0 2024-04-25T00:55:25.793 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.793+0000 7f4aeeffd640 1 -- 172.21.15.112:0/2992416278 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f4ae8055bd0 con 0x7f4af8072df0 2024-04-25T00:55:25.794 INFO:teuthology.misc.health.smithi112.stdout: 2024-04-25T00:55:25.794 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":12,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:23.522624+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:25.794 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-04-25T00:55:25.793+0000 7fc16c6dd640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T00:55:25.797 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.793+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4adc067830 msgr2=0x7f4adc069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:25.797 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.793+0000 7f4b00392640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4adc067830 0x7f4adc069cf0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4af8145f10 tx=0x7f4af0008040 comp rx=0 tx=0).stop 2024-04-25T00:55:25.797 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.793+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af8072df0 msgr2=0x7f4af81e7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:25.797 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.793+0000 7f4b00392640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af8072df0 0x7f4af81e7880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c440 tx=0x7f4ae800c900 comp rx=0 tx=0).stop 2024-04-25T00:55:25.798 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.797+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 shutdown_connections 2024-04-25T00:55:25.798 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.797+0000 7f4b00392640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4af80b3090 0x7f4af8140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.798 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.797+0000 7f4b00392640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4af8072df0 0x7f4af81e7880 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.798 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.797+0000 7f4b00392640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4adc067830 0x7f4adc069cf0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.798 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.797+0000 7f4b00392640 1 --2- 172.21.15.112:0/2992416278 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4af806bb80 0x7f4af8145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:25.798 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.797+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 >> 172.21.15.112:0/2992416278 conn(0x7f4af80b9430 msgr2=0x7f4af80b86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:25.798 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.797+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 shutdown_connections 2024-04-25T00:55:25.798 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:25.797+0000 7f4b00392640 1 -- 172.21.15.112:0/2992416278 wait complete. 2024-04-25T00:55:25.814 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T00:55:25.832 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-04-25T00:55:25.829+0000 7f2a2ce79640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T00:55:25.834 INFO:tasks.ceph.osd.1.smithi112.stderr:2024-04-25T00:55:25.833+0000 7fc164d24640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:55:25.874 INFO:tasks.ceph.osd.0.smithi112.stderr:2024-04-25T00:55:25.873+0000 7f2a254c0640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:55:32.620 DEBUG:teuthology.orchestra.run.smithi112:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:32.742 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a76b8640 1 Processor -- start 2024-04-25T00:55:32.742 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a76b8640 1 -- start start 2024-04-25T00:55:32.743 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a76b8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a006bb80 0x7f55a00b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:32.743 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a76b8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f55a0141390 0x7f55a0141840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:32.743 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a76b8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55a00b2520 con 0x7f55a0144820 2024-04-25T00:55:32.743 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a76b8640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f55a0141d80 con 0x7f55a0141390 2024-04-25T00:55:32.743 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a76b8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55a006bf60 con 0x7f55a006bb80 2024-04-25T00:55:32.743 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a5454640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a006bb80 0x7f55a00b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:32.743 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a5454640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a006bb80 0x7f55a00b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:56872/0 (socket says 172.21.15.112:56872) 2024-04-25T00:55:32.744 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a5454640 1 -- 172.21.15.112:0/928916449 learned_addr learned my addr 172.21.15.112:0/928916449 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:32.744 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a5c55640 1 --2- 172.21.15.112:0/928916449 >> v2:172.21.15.187:3300/0 conn(0x7f55a0141390 0x7f55a0141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:32.744 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a5454640 1 -- 172.21.15.112:0/928916449 >> v2:172.21.15.187:3300/0 conn(0x7f55a0141390 msgr2=0x7f55a0141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:32.744 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a5454640 1 --2- 172.21.15.112:0/928916449 >> v2:172.21.15.187:3300/0 conn(0x7f55a0141390 0x7f55a0141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.744 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a5454640 1 -- 172.21.15.112:0/928916449 >> v1:172.21.15.187:6789/0 conn(0x7f55a0144820 legacy=0x7f55a0144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:32.744 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.741+0000 7f55a5454640 1 -- 172.21.15.112:0/928916449 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a0144320 con 0x7f55a006bb80 2024-04-25T00:55:32.746 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a5454640 1 --2- 172.21.15.112:0/928916449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a006bb80 0x7f55a00b1fe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5598009e50 tx=0x7f559801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52bdb7227f8ef2f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:32.746 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f5593fff640 1 -- 172.21.15.112:0/928916449 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980a5750 con 0x7f55a006bb80 2024-04-25T00:55:32.746 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f5593fff640 1 -- 172.21.15.112:0/928916449 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55980a58f0 con 0x7f55a006bb80 2024-04-25T00:55:32.747 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f5593fff640 1 -- 172.21.15.112:0/928916449 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980a5cc0 con 0x7f55a006bb80 2024-04-25T00:55:32.747 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- 172.21.15.112:0/928916449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a006bb80 msgr2=0x7f55a00b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:32.748 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 --2- 172.21.15.112:0/928916449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a006bb80 0x7f55a00b1fe0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5598009e50 tx=0x7f559801c9a0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.748 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- 172.21.15.112:0/928916449 shutdown_connections 2024-04-25T00:55:32.748 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 --2- 172.21.15.112:0/928916449 >> v2:172.21.15.187:3300/0 conn(0x7f55a0141390 0x7f55a0141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.748 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 --2- 172.21.15.112:0/928916449 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a006bb80 0x7f55a00b1fe0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.748 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- 172.21.15.112:0/928916449 >> 172.21.15.112:0/928916449 conn(0x7f55a00b9430 msgr2=0x7f55a006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:32.748 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- 172.21.15.112:0/928916449 shutdown_connections 2024-04-25T00:55:32.748 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- 172.21.15.112:0/928916449 wait complete. 2024-04-25T00:55:32.750 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 Processor -- start 2024-04-25T00:55:32.750 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- start start 2024-04-25T00:55:32.750 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a006bb80 0x7f55a00b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:32.750 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a0141390 0x7f55a00b7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55a0144820 0x7f55a013a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f55a01513e0 con 0x7f55a006bb80 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f55a00b4940 con 0x7f55a0144820 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a76b8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f55a0151560 con 0x7f55a0141390 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a5454640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a006bb80 0x7f55a00b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a5454640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a006bb80 0x7f55a00b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33516/0 (socket says 172.21.15.112:33516) 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.745+0000 7f55a5454640 1 -- 172.21.15.112:0/3113150992 learned_addr learned my addr 172.21.15.112:0/3113150992 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a5c55640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55a0144820 0x7f55a013a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a4c53640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a0141390 0x7f55a00b7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a5454640 1 -- 172.21.15.112:0/3113150992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a0141390 msgr2=0x7f55a00b7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a5454640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a0141390 0x7f55a00b7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a5454640 1 -- 172.21.15.112:0/3113150992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55a0144820 msgr2=0x7f55a013a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:32.751 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a5454640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55a0144820 0x7f55a013a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.752 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a5454640 1 -- 172.21.15.112:0/3113150992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a00b8710 con 0x7f55a006bb80 2024-04-25T00:55:32.752 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a5454640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a006bb80 0x7f55a00b3ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f559801a5d0 tx=0x7f5598019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:32.752 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f5591ffb640 1 -- 172.21.15.112:0/3113150992 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980bc020 con 0x7f55a006bb80 2024-04-25T00:55:32.752 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f5591ffb640 1 -- 172.21.15.112:0/3113150992 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55980b8690 con 0x7f55a006bb80 2024-04-25T00:55:32.752 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a013f500 con 0x7f55a006bb80 2024-04-25T00:55:32.752 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f5591ffb640 1 -- 172.21.15.112:0/3113150992 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55980b8a80 con 0x7f55a006bb80 2024-04-25T00:55:32.752 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a00b8980 con 0x7f55a006bb80 2024-04-25T00:55:32.753 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55a0068b90 con 0x7f55a006bb80 2024-04-25T00:55:32.753 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f5591ffb640 1 -- 172.21.15.112:0/3113150992 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f55980bf020 con 0x7f55a006bb80 2024-04-25T00:55:32.753 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f5591ffb640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f55840678f0 0x7f5584069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:32.753 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f5591ffb640 1 -- 172.21.15.112:0/3113150992 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f55981204b0 con 0x7f55a006bb80 2024-04-25T00:55:32.753 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a4c53640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f55840678f0 0x7f5584069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:32.753 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.749+0000 7f55a4c53640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f55840678f0 0x7f5584069db0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f55a013eda0 tx=0x7f5594009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:32.757 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.753+0000 7f5591ffb640 1 -- 172.21.15.112:0/3113150992 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f55980c6070 con 0x7f55a006bb80 2024-04-25T00:55:32.919 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.917+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55a014dac0 con 0x7f55a006bb80 2024-04-25T00:55:32.920 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.917+0000 7f5591ffb640 1 -- 172.21.15.112:0/3113150992 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f5598016620 con 0x7f55a006bb80 2024-04-25T00:55:32.920 INFO:teuthology.misc.health.smithi112.stdout: 2024-04-25T00:55:32.921 INFO:teuthology.misc.health.smithi112.stdout:{"epoch":16,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:29.678758+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:32.922 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f55840678f0 msgr2=0x7f5584069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f55840678f0 0x7f5584069db0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f55a013eda0 tx=0x7f5594009040 comp rx=0 tx=0).stop 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a006bb80 msgr2=0x7f55a00b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a006bb80 0x7f55a00b3ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f559801a5d0 tx=0x7f5598019740 comp rx=0 tx=0).stop 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 shutdown_connections 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f55840678f0 0x7f5584069db0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f55a0144820 0x7f55a013a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f55a0141390 0x7f55a00b7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 --2- 172.21.15.112:0/3113150992 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55a006bb80 0x7f55a00b3ea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 >> 172.21.15.112:0/3113150992 conn(0x7f55a00b9430 msgr2=0x7f55a00b6080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 shutdown_connections 2024-04-25T00:55:32.923 INFO:teuthology.misc.health.smithi112.stderr:2024-04-25T00:55:32.921+0000 7f55a76b8640 1 -- 172.21.15.112:0/3113150992 wait complete. 2024-04-25T00:55:32.939 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T00:55:32.939 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T00:55:32.939 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:55:32.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:55:32.941 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:55:32.941 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:55:32.942 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:55:32.944 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:55:32.944 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T00:55:32.945 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:33.019 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-04-25T00:55:33.042 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T00:55:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.053+0000 7f23e7943640 1 Processor -- start 2024-04-25T00:55:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.053+0000 7f23e7943640 1 -- start start 2024-04-25T00:55:33.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e00671b0 0x7f23e00b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23e0071c40 con 0x7f23e00b48c0 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23e0067590 con 0x7f23e0141880 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23e0071680 con 0x7f23e00671b0 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e4ede640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f23e00b48c0 0x7f23e0141170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52498/0 (socket says 172.21.15.112:52498) 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e4ede640 1 -- 172.21.15.112:0/2777699862 learned_addr learned my addr 172.21.15.112:0/2777699862 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e56df640 1 --2- 172.21.15.112:0/2777699862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e00671b0 0x7f23e00b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23d7fff640 1 -- 172.21.15.112:0/2777699862 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567092896 0 0) 0x7f23e0071c40 con 0x7f23e00b48c0 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23d7fff640 1 -- 172.21.15.112:0/2777699862 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23cc003530 con 0x7f23e00b48c0 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23d7fff640 1 -- 172.21.15.112:0/2777699862 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739458084 0 0) 0x7f23e0067590 con 0x7f23e0141880 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e56df640 1 -- 172.21.15.112:0/2777699862 >> v1:172.21.15.187:6789/0 conn(0x7f23e0141880 legacy=0x7f23e0151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e56df640 1 -- 172.21.15.112:0/2777699862 >> v1:172.21.15.112:6789/0 conn(0x7f23e00b48c0 legacy=0x7f23e0141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e56df640 1 -- 172.21.15.112:0/2777699862 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e0071eb0 con 0x7f23e00671b0 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e56df640 1 --2- 172.21.15.112:0/2777699862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e00671b0 0x7f23e00b58a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097f0 tx=0x7f23d001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55130d23d73c9645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23d7fff640 1 -- 172.21.15.112:0/2777699862 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d00b0070 con 0x7f23e00671b0 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23d7fff640 1 -- 172.21.15.112:0/2777699862 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23d00ab600 con 0x7f23e00671b0 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23d7fff640 1 -- 172.21.15.112:0/2777699862 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d00ab9d0 con 0x7f23e00671b0 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 -- 172.21.15.112:0/2777699862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e00671b0 msgr2=0x7f23e00b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 --2- 172.21.15.112:0/2777699862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e00671b0 0x7f23e00b58a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097f0 tx=0x7f23d001c450 comp rx=0 tx=0).stop 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 -- 172.21.15.112:0/2777699862 shutdown_connections 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 --2- 172.21.15.112:0/2777699862 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e00671b0 0x7f23e00b58a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 -- 172.21.15.112:0/2777699862 >> 172.21.15.112:0/2777699862 conn(0x7f23e006cf00 msgr2=0x7f23e006d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 -- 172.21.15.112:0/2777699862 shutdown_connections 2024-04-25T00:55:33.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.057+0000 7f23e7943640 1 -- 172.21.15.112:0/2777699862 wait complete. 2024-04-25T00:55:33.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 Processor -- start 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 -- start start 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e00671b0 0x7f23e00b9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e00b48c0 0x7f23e01444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e0141880 0x7f23e01364e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f23e0155da0 con 0x7f23e00671b0 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f23e0155c20 con 0x7f23e00b48c0 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f23e0155f20 con 0x7f23e0141880 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e00671b0 0x7f23e00b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e00671b0 0x7f23e00b9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38440/0 (socket says 172.21.15.112:38440) 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 -- 172.21.15.112:0/1832940652 learned_addr learned my addr 172.21.15.112:0/1832940652 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e5ee0640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e0141880 0x7f23e01364e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e4ede640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e00b48c0 0x7f23e01444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 -- 172.21.15.112:0/1832940652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e0141880 msgr2=0x7f23e01364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e0141880 0x7f23e01364e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 -- 172.21.15.112:0/1832940652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e00b48c0 msgr2=0x7f23e01444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e00b48c0 0x7f23e01444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 -- 172.21.15.112:0/1832940652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e01426a0 con 0x7f23e00671b0 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e56df640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e00671b0 0x7f23e00b9100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097f0 tx=0x7f23d001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23d5ffb640 1 -- 172.21.15.112:0/1832940652 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d00bb020 con 0x7f23e00671b0 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23d5ffb640 1 -- 172.21.15.112:0/1832940652 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23d00b1040 con 0x7f23e00671b0 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e0135d70 con 0x7f23e00671b0 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23d5ffb640 1 -- 172.21.15.112:0/1832940652 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d00b0050 con 0x7f23e00671b0 2024-04-25T00:55:33.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e0067cd0 con 0x7f23e00671b0 2024-04-25T00:55:33.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.061+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e004f580 con 0x7f23e00671b0 2024-04-25T00:55:33.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.065+0000 7f23d5ffb640 1 -- 172.21.15.112:0/1832940652 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23d00be070 con 0x7f23e00671b0 2024-04-25T00:55:33.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.065+0000 7f23d5ffb640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f23c40678f0 0x7f23c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.065+0000 7f23d5ffb640 1 -- 172.21.15.112:0/1832940652 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f23d011ed30 con 0x7f23e00671b0 2024-04-25T00:55:33.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.065+0000 7f23e4ede640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f23c40678f0 0x7f23c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.065+0000 7f23e4ede640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f23c40678f0 0x7f23c4069db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23dc007fd0 tx=0x7f23dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.069+0000 7f23d5ffb640 1 -- 172.21.15.112:0/1832940652 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f23d00160d0 con 0x7f23e00671b0 2024-04-25T00:55:33.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.221+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23e00b3dd0 con 0x7f23e00671b0 2024-04-25T00:55:33.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.221+0000 7f23d5ffb640 1 -- 172.21.15.112:0/1832940652 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f23d00ef600 con 0x7f23e00671b0 2024-04-25T00:55:33.224 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:33.225 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:29.678758+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f23c40678f0 msgr2=0x7f23c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f23c40678f0 0x7f23c4069db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23dc007fd0 tx=0x7f23dc009040 comp rx=0 tx=0).stop 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e00671b0 msgr2=0x7f23e00b9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e00671b0 0x7f23e00b9100 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097f0 tx=0x7f23d001ac90 comp rx=0 tx=0).stop 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 shutdown_connections 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f23e0141880 0x7f23e01364e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f23c40678f0 0x7f23c4069db0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f23e00b48c0 0x7f23e01444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 --2- 172.21.15.112:0/1832940652 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23e00671b0 0x7f23e00b9100 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 >> 172.21.15.112:0/1832940652 conn(0x7f23e006cf00 msgr2=0x7f23e00b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:33.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 shutdown_connections 2024-04-25T00:55:33.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.225+0000 7f23e7943640 1 -- 172.21.15.112:0/1832940652 wait complete. 2024-04-25T00:55:33.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:33.244 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.361+0000 7f16c9bf7640 1 Processor -- start 2024-04-25T00:55:33.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.361+0000 7f16c9bf7640 1 -- start start 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c409b800 0x7f16c4141350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16c4141890 0x7f16c4151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16c40b9410 con 0x7f16c40b9030 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f16c409b3c0 con 0x7f16c4141890 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16c40b9990 con 0x7f16c409b800 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c37fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f16c40b9030 0x7f16c409acb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52508/0 (socket says 172.21.15.112:52508) 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c37fe640 1 -- 172.21.15.112:0/731492240 learned_addr learned my addr 172.21.15.112:0/731492240 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c3fff640 1 --2- 172.21.15.112:0/731492240 >> v2:172.21.15.112:3300/0 conn(0x7f16c4141890 0x7f16c4151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c27fc640 1 -- 172.21.15.112:0/731492240 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486318480 0 0) 0x7f16c40b9410 con 0x7f16c40b9030 2024-04-25T00:55:33.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c27fc640 1 -- 172.21.15.112:0/731492240 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16ac003530 con 0x7f16c40b9030 2024-04-25T00:55:33.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c2ffd640 1 --2- 172.21.15.112:0/731492240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c409b800 0x7f16c4141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c3fff640 1 -- 172.21.15.112:0/731492240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c409b800 msgr2=0x7f16c4141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:33.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c3fff640 1 --2- 172.21.15.112:0/731492240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c409b800 0x7f16c4141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c3fff640 1 -- 172.21.15.112:0/731492240 >> v1:172.21.15.112:6789/0 conn(0x7f16c40b9030 legacy=0x7f16c409acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c27fc640 1 -- 172.21.15.112:0/731492240 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3948972611 0 0) 0x7f16ac003530 con 0x7f16c40b9030 2024-04-25T00:55:33.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c3fff640 1 -- 172.21.15.112:0/731492240 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c40baa90 con 0x7f16c4141890 2024-04-25T00:55:33.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c3fff640 1 --2- 172.21.15.112:0/731492240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c4151ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009d90 tx=0x7f16bc01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97af481d8a945829 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c27fc640 1 -- 172.21.15.112:0/731492240 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8001680 con 0x7f16c4141890 2024-04-25T00:55:33.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c27fc640 1 -- 172.21.15.112:0/731492240 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc0a6500 con 0x7f16c4141890 2024-04-25T00:55:33.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c27fc640 1 -- 172.21.15.112:0/731492240 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0a68d0 con 0x7f16c4141890 2024-04-25T00:55:33.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 -- 172.21.15.112:0/731492240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 msgr2=0x7f16c4151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/731492240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c4151ce0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009d90 tx=0x7f16bc01c800 comp rx=0 tx=0).stop 2024-04-25T00:55:33.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 -- 172.21.15.112:0/731492240 shutdown_connections 2024-04-25T00:55:33.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/731492240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c4151ce0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/731492240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c409b800 0x7f16c4141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.365+0000 7f16c9bf7640 1 -- 172.21.15.112:0/731492240 >> 172.21.15.112:0/731492240 conn(0x7f16c409eaf0 msgr2=0x7f16c409ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:33.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- 172.21.15.112:0/731492240 shutdown_connections 2024-04-25T00:55:33.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- 172.21.15.112:0/731492240 wait complete. 2024-04-25T00:55:33.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 Processor -- start 2024-04-25T00:55:33.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- start start 2024-04-25T00:55:33.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c409b800 0x7f16c413b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c40b9030 0x7f16c4138a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c41dfce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16c40b9e20 con 0x7f16c4141890 2024-04-25T00:55:33.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16c409c140 con 0x7f16c409b800 2024-04-25T00:55:33.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16c41df820 con 0x7f16c40b9030 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c41dfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c41dfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38456/0 (socket says 172.21.15.112:38456) 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 -- 172.21.15.112:0/332290112 learned_addr learned my addr 172.21.15.112:0/332290112 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c37fe640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c409b800 0x7f16c413b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 -- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c40b9030 msgr2=0x7f16c4138a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c2ffd640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c40b9030 0x7f16c4138a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c40b9030 0x7f16c4138a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 -- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c409b800 msgr2=0x7f16c413b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c409b800 0x7f16c413b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 -- 172.21.15.112:0/332290112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c41e3810 con 0x7f16c4141890 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c37fe640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c409b800 0x7f16c413b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c3fff640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c41dfce0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0b0040 tx=0x7f16bc0195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16abfff640 1 -- 172.21.15.112:0/332290112 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc009450 con 0x7f16c4141890 2024-04-25T00:55:33.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16abfff640 1 -- 172.21.15.112:0/332290112 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc0c0070 con 0x7f16c4141890 2024-04-25T00:55:33.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c41e3af0 con 0x7f16c4141890 2024-04-25T00:55:33.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16abfff640 1 -- 172.21.15.112:0/332290112 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0ba8d0 con 0x7f16c4141890 2024-04-25T00:55:33.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c40b6370 con 0x7f16c4141890 2024-04-25T00:55:33.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.369+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c41df9a0 con 0x7f16c4141890 2024-04-25T00:55:33.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.373+0000 7f16abfff640 1 -- 172.21.15.112:0/332290112 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f16bc0bac80 con 0x7f16c4141890 2024-04-25T00:55:33.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.373+0000 7f16abfff640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16b00678c0 0x7f16b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.373+0000 7f16abfff640 1 -- 172.21.15.112:0/332290112 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f16bc121ae0 con 0x7f16c4141890 2024-04-25T00:55:33.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.373+0000 7f16c37fe640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16b00678c0 0x7f16b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.373+0000 7f16c37fe640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16b00678c0 0x7f16b0069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16b80079c0 tx=0x7f16b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.377+0000 7f16abfff640 1 -- 172.21.15.112:0/332290112 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f16bc016480 con 0x7f16c4141890 2024-04-25T00:55:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f16c40b6fe0 con 0x7f16c4141890 2024-04-25T00:55:33.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16abfff640 1 -- 172.21.15.112:0/332290112 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f16bc0c43c0 con 0x7f16c4141890 2024-04-25T00:55:33.539 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:33.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16b00678c0 msgr2=0x7f16b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16b00678c0 0x7f16b0069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16b80079c0 tx=0x7f16b8009040 comp rx=0 tx=0).stop 2024-04-25T00:55:33.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 msgr2=0x7f16c41dfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c41dfce0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0b0040 tx=0x7f16bc0195a0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 shutdown_connections 2024-04-25T00:55:33.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16c4141890 0x7f16c41dfce0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16c40b9030 0x7f16c4138a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16b00678c0 0x7f16b0069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 --2- 172.21.15.112:0/332290112 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16c409b800 0x7f16c413b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 >> 172.21.15.112:0/332290112 conn(0x7f16c409eaf0 msgr2=0x7f16c40bcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:33.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 shutdown_connections 2024-04-25T00:55:33.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.537+0000 7f16c9bf7640 1 -- 172.21.15.112:0/332290112 wait complete. 2024-04-25T00:55:33.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:55:33.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.673+0000 7fa10ffd0640 1 Processor -- start 2024-04-25T00:55:33.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.673+0000 7fa10ffd0640 1 -- start start 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10ffd0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa10806c520 0x7fa1080727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10ffd0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1080b47d0 0x7fa1080b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10ffd0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1080b5300 con 0x7fa108141110 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10ffd0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa10806c900 con 0x7fa10806c520 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10ffd0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa1080ae670 con 0x7fa1080b47d0 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10e56d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa108141110 0x7fa1081415a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52510/0 (socket says 172.21.15.112:52510) 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10e56d640 1 -- 172.21.15.112:0/1500795583 learned_addr learned my addr 172.21.15.112:0/1500795583 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10d56b640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1080b47d0 0x7fa1080b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10dd6c640 1 --2- 172.21.15.112:0/1500795583 >> v2:172.21.15.187:3300/0 conn(0x7fa10806c520 0x7fa1080727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3722584446 0 0) 0x7fa1080b5300 con 0x7fa108141110 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0f0003530 con 0x7fa108141110 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 559405348 0 0) 0x7fa0f0003530 con 0x7fa108141110 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1080b47d0 msgr2=0x7fa1080b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1080b47d0 0x7fa1080b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 >> v2:172.21.15.187:3300/0 conn(0x7fa10806c520 msgr2=0x7fa1080727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 --2- 172.21.15.112:0/1500795583 >> v2:172.21.15.187:3300/0 conn(0x7fa10806c520 0x7fa1080727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1080ae950 con 0x7fa108141110 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7fa0f8001680 con 0x7fa108141110 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 >> v1:172.21.15.112:6789/0 conn(0x7fa108141110 legacy=0x7fa1081415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1080b47d0 0x7fa0f00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0003f40 0x7fa0f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f0004d50 0x7fa0f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa10806c900 con 0x7fa0f0003f40 2024-04-25T00:55:33.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1080ae670 con 0x7fa0f0004d50 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f000ba20 con 0x7fa1080b47d0 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10d56b640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0003f40 0x7fa0f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10dd6c640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1080b47d0 0x7fa0f00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10e56d640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f0004d50 0x7fa0f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10d56b640 1 -- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1080b47d0 msgr2=0x7fa0f00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10d56b640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1080b47d0 0x7fa0f00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10d56b640 1 -- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f0004d50 msgr2=0x7fa0f00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10d56b640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f0004d50 0x7fa0f00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10d56b640 1 -- 172.21.15.112:0/1500795583 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f0003530 con 0x7fa0f0003f40 2024-04-25T00:55:33.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.677+0000 7fa10e56d640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0f0004d50 0x7fa0f00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:33.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10d56b640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0003f40 0x7fa0f0004810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009b10 tx=0x7fa0fc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef1f25920f6fd7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0096b0 con 0x7fa0f0003f40 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0fc0a6450 con 0x7fa0f0003f40 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10cd6a640 1 -- 172.21.15.112:0/1500795583 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc0a6840 con 0x7fa0f0003f40 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- 172.21.15.112:0/1500795583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0003f40 msgr2=0x7fa0f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 --2- 172.21.15.112:0/1500795583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0f0003f40 0x7fa0f0004810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc009b10 tx=0x7fa0fc01ca70 comp rx=0 tx=0).stop 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 -- 172.21.15.112:0/1500795583 reap_dead start 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- 172.21.15.112:0/1500795583 shutdown_connections 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- 172.21.15.112:0/1500795583 >> 172.21.15.112:0/1500795583 conn(0x7fa1080b5980 msgr2=0x7fa1080b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- 172.21.15.112:0/1500795583 shutdown_connections 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- 172.21.15.112:0/1500795583 wait complete. 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 Processor -- start 2024-04-25T00:55:33.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- start start 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0f0003f40 0x7fa108146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108144160 0x7fa1081431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1081e3820 0x7fa1081e3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1081526a0 con 0x7fa1081e3820 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa108152520 con 0x7fa108144160 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa108152820 con 0x7fa0f0003f40 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1081e3820 0x7fa1081e3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1081e3820 0x7fa1081e3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38484/0 (socket says 172.21.15.112:38484) 2024-04-25T00:55:33.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 -- 172.21.15.112:0/2761647680 learned_addr learned my addr 172.21.15.112:0/2761647680 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10dd6c640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0f0003f40 0x7fa108146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10d56b640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108144160 0x7fa1081431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 -- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0f0003f40 msgr2=0x7fa108146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0f0003f40 0x7fa108146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 -- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108144160 msgr2=0x7fa1081431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108144160 0x7fa1081431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 -- 172.21.15.112:0/2761647680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10813e110 con 0x7fa1081e3820 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10dd6c640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0f0003f40 0x7fa108146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10e56d640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1081e3820 0x7fa1081e3ca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c10 tx=0x7fa0f800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa1067fc640 1 -- 172.21.15.112:0/2761647680 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80075a0 con 0x7fa1081e3820 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa1067fc640 1 -- 172.21.15.112:0/2761647680 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f80047f0 con 0x7fa1081e3820 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa108143ce0 con 0x7fa1081e3820 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa1067fc640 1 -- 172.21.15.112:0/2761647680 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8004be0 con 0x7fa1081e3820 2024-04-25T00:55:33.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.681+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1081428c0 con 0x7fa1081e3820 2024-04-25T00:55:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.685+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa10804f580 con 0x7fa1081e3820 2024-04-25T00:55:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.685+0000 7fa1067fc640 1 -- 172.21.15.112:0/2761647680 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa0f801b020 con 0x7fa1081e3820 2024-04-25T00:55:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.685+0000 7fa1067fc640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa0f40678c0 0x7fa0f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.685+0000 7fa1067fc640 1 -- 172.21.15.112:0/2761647680 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fa0f8085380 con 0x7fa1081e3820 2024-04-25T00:55:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.685+0000 7fa10dd6c640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa0f40678c0 0x7fa0f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:33.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.685+0000 7fa10dd6c640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa0f40678c0 0x7fa0f4069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa1000074d0 tx=0x7fa10000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:33.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.689+0000 7fa1067fc640 1 -- 172.21.15.112:0/2761647680 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa0f80562a0 con 0x7fa1081e3820 2024-04-25T00:55:33.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.885+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa108142e70 con 0x7fa1081e3820 2024-04-25T00:55:33.888 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.885+0000 7fa1067fc640 1 -- 172.21.15.112:0/2761647680 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fa0f8055ba0 con 0x7fa1081e3820 2024-04-25T00:55:33.889 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:33.889 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:55:33.889 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 2 2024-04-25T00:55:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa0f40678c0 msgr2=0x7fa0f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa0f40678c0 0x7fa0f4069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa1000074d0 tx=0x7fa10000b040 comp rx=0 tx=0).stop 2024-04-25T00:55:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1081e3820 msgr2=0x7fa1081e3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1081e3820 0x7fa1081e3ca0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000c10 tx=0x7fa0f800d9d0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 shutdown_connections 2024-04-25T00:55:33.891 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1081e3820 0x7fa1081e3ca0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa0f40678c0 0x7fa0f4069d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa108144160 0x7fa1081431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 --2- 172.21.15.112:0/2761647680 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0f0003f40 0x7fa108146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 >> 172.21.15.112:0/2761647680 conn(0x7fa1080b5980 msgr2=0x7fa108147500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 shutdown_connections 2024-04-25T00:55:33.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:33.889+0000 7fa10ffd0640 1 -- 172.21.15.112:0/2761647680 wait complete. 2024-04-25T00:55:35.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:55:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.032+0000 7f295e52f640 1 Processor -- start 2024-04-25T00:55:36.038 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.032+0000 7f295e52f640 1 -- start start 2024-04-25T00:55:36.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f295e52f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f295806c520 0x7f29580727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f295e52f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29580b47d0 0x7f29580b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f295e52f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29580b5300 con 0x7f2958141110 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f295e52f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f295806c900 con 0x7f29580b47d0 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f295e52f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f29580ae670 con 0x7f295806c520 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f29577fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29580b47d0 0x7f29580b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f29577fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29580b47d0 0x7f29580b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38494/0 (socket says 172.21.15.112:38494) 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f29577fe640 1 -- 172.21.15.112:0/2717185214 learned_addr learned my addr 172.21.15.112:0/2717185214 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f2957fff640 1 --2- 172.21.15.112:0/2717185214 >> v2:172.21.15.187:3300/0 conn(0x7f295806c520 0x7f29580727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f29577fe640 1 -- 172.21.15.112:0/2717185214 >> v2:172.21.15.187:3300/0 conn(0x7f295806c520 msgr2=0x7f29580727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f29577fe640 1 --2- 172.21.15.112:0/2717185214 >> v2:172.21.15.187:3300/0 conn(0x7f295806c520 0x7f29580727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f29577fe640 1 -- 172.21.15.112:0/2717185214 >> v1:172.21.15.187:6789/0 conn(0x7f2958141110 legacy=0x7f2958141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f29577fe640 1 -- 172.21.15.112:0/2717185214 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29580ae950 con 0x7f29580b47d0 2024-04-25T00:55:36.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f2957fff640 1 --2- 172.21.15.112:0/2717185214 >> v2:172.21.15.187:3300/0 conn(0x7f295806c520 0x7f29580727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f29577fe640 1 --2- 172.21.15.112:0/2717185214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b4c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29580b9bc0 tx=0x7f294001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2c1d5f41cb2b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f2956ffd640 1 -- 172.21.15.112:0/2717185214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400b0070 con 0x7f29580b47d0 2024-04-25T00:55:36.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f2956ffd640 1 -- 172.21.15.112:0/2717185214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29400ab600 con 0x7f29580b47d0 2024-04-25T00:55:36.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.036+0000 7f2956ffd640 1 -- 172.21.15.112:0/2717185214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400ab9d0 con 0x7f29580b47d0 2024-04-25T00:55:36.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- 172.21.15.112:0/2717185214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 msgr2=0x7f29580b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 --2- 172.21.15.112:0/2717185214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b4c30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29580b9bc0 tx=0x7f294001c450 comp rx=0 tx=0).stop 2024-04-25T00:55:36.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- 172.21.15.112:0/2717185214 shutdown_connections 2024-04-25T00:55:36.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 --2- 172.21.15.112:0/2717185214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b4c30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 --2- 172.21.15.112:0/2717185214 >> v2:172.21.15.187:3300/0 conn(0x7f295806c520 0x7f29580727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- 172.21.15.112:0/2717185214 >> 172.21.15.112:0/2717185214 conn(0x7f29580b5980 msgr2=0x7f29580b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:36.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- 172.21.15.112:0/2717185214 shutdown_connections 2024-04-25T00:55:36.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- 172.21.15.112:0/2717185214 wait complete. 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 Processor -- start 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- start start 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f295806c520 0x7f29580b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2958141110 0x7f2958142a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29581548c0 con 0x7f29580b47d0 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2958154740 con 0x7f295806c520 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2958154a40 con 0x7f2958141110 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38502/0 (socket says 172.21.15.112:38502) 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 -- 172.21.15.112:0/3386237401 learned_addr learned my addr 172.21.15.112:0/3386237401 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f2957fff640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f295806c520 0x7f29580b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295cacc640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2958141110 0x7f2958142a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 -- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2958141110 msgr2=0x7f2958142a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2958141110 0x7f2958142a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 -- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f295806c520 msgr2=0x7f29580b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f295806c520 0x7f29580b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 -- 172.21.15.112:0/3386237401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29580b9040 con 0x7f29580b47d0 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295cacc640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2958141110 0x7f2958142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f29577fe640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b8b00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29400ae040 tx=0x7f29400191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f2954ff9640 1 -- 172.21.15.112:0/3386237401 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400b0040 con 0x7f29580b47d0 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f2954ff9640 1 -- 172.21.15.112:0/3386237401 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29400b9070 con 0x7f29580b47d0 2024-04-25T00:55:36.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29580727d0 con 0x7f29580b47d0 2024-04-25T00:55:36.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f2954ff9640 1 -- 172.21.15.112:0/3386237401 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400b48a0 con 0x7f29580b47d0 2024-04-25T00:55:36.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.040+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2958143fd0 con 0x7f29580b47d0 2024-04-25T00:55:36.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.044+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295804f580 con 0x7f29580b47d0 2024-04-25T00:55:36.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.044+0000 7f2954ff9640 1 -- 172.21.15.112:0/3386237401 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f29400ab4c0 con 0x7f29580b47d0 2024-04-25T00:55:36.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.044+0000 7f2954ff9640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29380678c0 0x7f2938069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.044+0000 7f2954ff9640 1 -- 172.21.15.112:0/3386237401 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f294011f660 con 0x7f29580b47d0 2024-04-25T00:55:36.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.044+0000 7f2957fff640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29380678c0 0x7f2938069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.044+0000 7f2957fff640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29380678c0 0x7f2938069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f294800a9a0 tx=0x7f2948008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.048+0000 7f2954ff9640 1 -- 172.21.15.112:0/3386237401 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f29580b9040 con 0x7f29580b47d0 2024-04-25T00:55:36.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.232+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2958073700 con 0x7f29580b47d0 2024-04-25T00:55:36.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.232+0000 7f2954ff9640 1 -- 172.21.15.112:0/3386237401 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2219 (secure 0 0 0) 0x7f29400160d0 con 0x7f29580b47d0 2024-04-25T00:55:36.237 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:36.237 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:55:36.237 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 3 2024-04-25T00:55:36.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29380678c0 msgr2=0x7f2938069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29380678c0 0x7f2938069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f294800a9a0 tx=0x7f2948008040 comp rx=0 tx=0).stop 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 msgr2=0x7f29580b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b8b00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29400ae040 tx=0x7f29400191f0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 shutdown_connections 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2958141110 0x7f2958142a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29580b47d0 0x7f29580b8b00 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29400ae040 tx=0x7f29400191f0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29380678c0 0x7f2938069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 --2- 172.21.15.112:0/3386237401 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f295806c520 0x7f29580b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 >> 172.21.15.112:0/3386237401 conn(0x7f29580b5980 msgr2=0x7f2958069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 shutdown_connections 2024-04-25T00:55:36.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.236+0000 7f295e52f640 1 -- 172.21.15.112:0/3386237401 wait complete. 2024-04-25T00:55:36.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:36.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.352+0000 7f5017b6f640 1 Processor -- start 2024-04-25T00:55:36.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.352+0000 7f5017b6f640 1 -- start start 2024-04-25T00:55:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5017b6f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 0x7f5010145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5017b6f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f50100b3ce0 0x7f501014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5017b6f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50100aeec0 con 0x7f50100b23f0 2024-04-25T00:55:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5017b6f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f50100af780 con 0x7f50100b3ce0 2024-04-25T00:55:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5017b6f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50100b4170 con 0x7f50100add70 2024-04-25T00:55:36.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501590b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f50100b23f0 0x7f5010072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52524/0 (socket says 172.21.15.112:52524) 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501590b640 1 -- 172.21.15.112:0/85142013 learned_addr learned my addr 172.21.15.112:0/85142013 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501510a640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 0x7f5010145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501610c640 1 --2- 172.21.15.112:0/85142013 >> v2:172.21.15.187:3300/0 conn(0x7f50100b3ce0 0x7f501014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390372698 0 0) 0x7f50100aeec0 con 0x7f50100b23f0 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ffc007940 con 0x7f50100b23f0 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1784381455 0 0) 0x7f4ffc007940 con 0x7f50100b23f0 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 msgr2=0x7f5010145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 0x7f5010145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 >> v2:172.21.15.187:3300/0 conn(0x7f50100b3ce0 msgr2=0x7f501014fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 --2- 172.21.15.112:0/85142013 >> v2:172.21.15.187:3300/0 conn(0x7f50100b3ce0 0x7f501014fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50100b27d0 con 0x7f50100b23f0 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f5008001a00 con 0x7f50100b23f0 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 >> v1:172.21.15.112:6789/0 conn(0x7f50100b23f0 legacy=0x7f5010072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ffc007dd0 0x7f4ffc0081f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 0x7f4ffc008c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ffc009160 0x7f4ffc0095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50100af780 con 0x7f4ffc007dd0 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50100b4170 con 0x7f4ffc009160 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4ffc00fde0 con 0x7f50100add70 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501590b640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ffc007dd0 0x7f4ffc0081f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501610c640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ffc009160 0x7f4ffc0095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501510a640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 0x7f4ffc008c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501590b640 1 -- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 msgr2=0x7f4ffc008c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501590b640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 0x7f4ffc008c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501590b640 1 -- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ffc009160 msgr2=0x7f4ffc0095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501590b640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4ffc009160 0x7f4ffc0095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501590b640 1 -- 172.21.15.112:0/85142013 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc007940 con 0x7f4ffc007dd0 2024-04-25T00:55:36.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.356+0000 7f501510a640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f50100add70 0x7f4ffc008c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:36.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f501590b640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ffc007dd0 0x7f4ffc0081f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f500800a950 tx=0x7f500801d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bc47184cd2247 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500800a4c0 con 0x7f4ffc007dd0 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50080ad650 con 0x7f4ffc007dd0 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5014909640 1 -- 172.21.15.112:0/85142013 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50080ada40 con 0x7f4ffc007dd0 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- 172.21.15.112:0/85142013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ffc007dd0 msgr2=0x7f4ffc0081f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 --2- 172.21.15.112:0/85142013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ffc007dd0 0x7f4ffc0081f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f500800a950 tx=0x7f500801d440 comp rx=0 tx=0).stop 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f501610c640 1 -- 172.21.15.112:0/85142013 reap_dead start 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- 172.21.15.112:0/85142013 shutdown_connections 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- 172.21.15.112:0/85142013 >> 172.21.15.112:0/85142013 conn(0x7f50100b5980 msgr2=0x7f50100b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- 172.21.15.112:0/85142013 shutdown_connections 2024-04-25T00:55:36.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- 172.21.15.112:0/85142013 wait complete. 2024-04-25T00:55:36.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 Processor -- start 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- start start 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ffc007dd0 0x7f5010141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50101df310 0x7f50101df790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f501013ed30 0x7f501013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f50100b5150 con 0x7f501013ed30 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f50100ae150 con 0x7f50101df310 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f5017b6f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f50100b52d0 con 0x7f4ffc007dd0 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f501610c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f501013ed30 0x7f501013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f501610c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f501013ed30 0x7f501013f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38516/0 (socket says 172.21.15.112:38516) 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.360+0000 7f501610c640 1 -- 172.21.15.112:0/783296406 learned_addr learned my addr 172.21.15.112:0/783296406 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501590b640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ffc007dd0 0x7f5010141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501510a640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50101df310 0x7f50101df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501610c640 1 -- 172.21.15.112:0/783296406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ffc007dd0 msgr2=0x7f5010141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501610c640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ffc007dd0 0x7f5010141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501610c640 1 -- 172.21.15.112:0/783296406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50101df310 msgr2=0x7f50101df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501610c640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50101df310 0x7f50101df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501610c640 1 -- 172.21.15.112:0/783296406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501009c220 con 0x7f501013ed30 2024-04-25T00:55:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501610c640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f501013ed30 0x7f501013f1b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f500c011280 tx=0x7f500c011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f50027fc640 1 -- 172.21.15.112:0/783296406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c019020 con 0x7f501013ed30 2024-04-25T00:55:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f50027fc640 1 -- 172.21.15.112:0/783296406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f500c004440 con 0x7f501013ed30 2024-04-25T00:55:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501009c4c0 con 0x7f501013ed30 2024-04-25T00:55:36.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f50027fc640 1 -- 172.21.15.112:0/783296406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c003860 con 0x7f501013ed30 2024-04-25T00:55:36.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f501013f6f0 con 0x7f501013ed30 2024-04-25T00:55:36.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f501004f580 con 0x7f501013ed30 2024-04-25T00:55:36.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f50027fc640 1 -- 172.21.15.112:0/783296406 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f500c01c020 con 0x7f501013ed30 2024-04-25T00:55:36.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f50027fc640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4ff40678c0 0x7f4ff4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f50027fc640 1 -- 172.21.15.112:0/783296406 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f500c085970 con 0x7f501013ed30 2024-04-25T00:55:36.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501590b640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4ff40678c0 0x7f4ff4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.364+0000 7f501590b640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4ff40678c0 0x7f4ff4069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f50080ad000 tx=0x7f5008001dd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.368+0000 7f50027fc640 1 -- 172.21.15.112:0/783296406 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f500c023030 con 0x7f501013ed30 2024-04-25T00:55:36.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.524+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f50100b60c0 con 0x7f501013ed30 2024-04-25T00:55:36.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f50027fc640 1 -- 172.21.15.112:0/783296406 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12932 (secure 0 0 0) 0x7f500c056890 con 0x7f501013ed30 2024-04-25T00:55:36.530 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:36.530 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:29.678758+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4ff40678c0 msgr2=0x7f4ff4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4ff40678c0 0x7f4ff4069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f50080ad000 tx=0x7f5008001dd0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f501013ed30 msgr2=0x7f501013f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f501013ed30 0x7f501013f1b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f500c011280 tx=0x7f500c011740 comp rx=0 tx=0).stop 2024-04-25T00:55:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 shutdown_connections 2024-04-25T00:55:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f501013ed30 0x7f501013f1b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4ff40678c0 0x7f4ff4069d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f50101df310 0x7f50101df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 --2- 172.21.15.112:0/783296406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4ffc007dd0 0x7f5010141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 >> 172.21.15.112:0/783296406 conn(0x7f50100b5980 msgr2=0x7f501006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:36.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 shutdown_connections 2024-04-25T00:55:36.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.528+0000 7f5017b6f640 1 -- 172.21.15.112:0/783296406 wait complete. 2024-04-25T00:55:36.548 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:36.548 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:36.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe051819640 1 Processor -- start 2024-04-25T00:55:36.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe051819640 1 -- start start 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe051819640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe04c147500 0x7fe04c147d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe051819640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe04c06c620 0x7fe04c151da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe051819640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe04c0b8440 con 0x7fe04c06e1d0 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe051819640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe04c141cc0 con 0x7fe04c06c620 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe051819640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe04c06cba0 con 0x7fe04c147500 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe04c06c620 0x7fe04c151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04affd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe04c06e1d0 0x7fe04c14b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52530/0 (socket says 172.21.15.112:52530) 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04affd640 1 -- 172.21.15.112:0/2379273604 learned_addr learned my addr 172.21.15.112:0/2379273604 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04a7fc640 1 --2- 172.21.15.112:0/2379273604 >> v2:172.21.15.187:3300/0 conn(0x7fe04c147500 0x7fe04c147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04b7fe640 1 -- 172.21.15.112:0/2379273604 >> v2:172.21.15.187:3300/0 conn(0x7fe04c147500 msgr2=0x7fe04c147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04b7fe640 1 --2- 172.21.15.112:0/2379273604 >> v2:172.21.15.187:3300/0 conn(0x7fe04c147500 0x7fe04c147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04b7fe640 1 -- 172.21.15.112:0/2379273604 >> v1:172.21.15.112:6789/0 conn(0x7fe04c06e1d0 legacy=0x7fe04c14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe049ffb640 1 -- 172.21.15.112:0/2379273604 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4283761431 0 0) 0x7fe04c0b8440 con 0x7fe04c06e1d0 2024-04-25T00:55:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04b7fe640 1 -- 172.21.15.112:0/2379273604 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c06e5b0 con 0x7fe04c06c620 2024-04-25T00:55:36.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.668+0000 7fe04a7fc640 1 --2- 172.21.15.112:0/2379273604 >> v2:172.21.15.187:3300/0 conn(0x7fe04c147500 0x7fe04c147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:36.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 --2- 172.21.15.112:0/2379273604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c06c620 0x7fe04c151da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe04c070040 tx=0x7fe04401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f96af02fd472dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe049ffb640 1 -- 172.21.15.112:0/2379273604 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0440b2070 con 0x7fe04c06c620 2024-04-25T00:55:36.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe049ffb640 1 -- 172.21.15.112:0/2379273604 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0440a58d0 con 0x7fe04c06c620 2024-04-25T00:55:36.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe049ffb640 1 -- 172.21.15.112:0/2379273604 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0440a5ca0 con 0x7fe04c06c620 2024-04-25T00:55:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- 172.21.15.112:0/2379273604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c06c620 msgr2=0x7fe04c151da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 --2- 172.21.15.112:0/2379273604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c06c620 0x7fe04c151da0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe04c070040 tx=0x7fe04401c810 comp rx=0 tx=0).stop 2024-04-25T00:55:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- 172.21.15.112:0/2379273604 shutdown_connections 2024-04-25T00:55:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 --2- 172.21.15.112:0/2379273604 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c06c620 0x7fe04c151da0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 --2- 172.21.15.112:0/2379273604 >> v2:172.21.15.187:3300/0 conn(0x7fe04c147500 0x7fe04c147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- 172.21.15.112:0/2379273604 >> 172.21.15.112:0/2379273604 conn(0x7fe04c0711b0 msgr2=0x7fe04c071590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- 172.21.15.112:0/2379273604 shutdown_connections 2024-04-25T00:55:36.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- 172.21.15.112:0/2379273604 wait complete. 2024-04-25T00:55:36.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 Processor -- start 2024-04-25T00:55:36.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- start start 2024-04-25T00:55:36.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe04c06c620 0x7fe04c1dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe04c06e1d0 0x7fe04c1420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c147500 0x7fe04c143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c148420 con 0x7fe04c147500 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe04c1482a0 con 0x7fe04c06c620 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe04c154b00 con 0x7fe04c06e1d0 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c147500 0x7fe04c143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c147500 0x7fe04c143640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38544/0 (socket says 172.21.15.112:38544) 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 -- 172.21.15.112:0/1876346514 learned_addr learned my addr 172.21.15.112:0/1876346514 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04a7fc640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe04c06e1d0 0x7fe04c1420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 -- 172.21.15.112:0/1876346514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe04c06e1d0 msgr2=0x7fe04c1420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe04c06e1d0 0x7fe04c1420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04affd640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe04c06c620 0x7fe04c1dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 -- 172.21.15.112:0/1876346514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe04c06c620 msgr2=0x7fe04c1dfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe04c06c620 0x7fe04c1dfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 -- 172.21.15.112:0/1876346514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe04c144890 con 0x7fe04c147500 2024-04-25T00:55:36.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe04b7fe640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c147500 0x7fe04c143640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0440b0040 tx=0x7fe0440195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe033fff640 1 -- 172.21.15.112:0/1876346514 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0440b2050 con 0x7fe04c147500 2024-04-25T00:55:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe033fff640 1 -- 172.21.15.112:0/1876346514 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0440be070 con 0x7fe04c147500 2024-04-25T00:55:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe04c142800 con 0x7fe04c147500 2024-04-25T00:55:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe033fff640 1 -- 172.21.15.112:0/1876346514 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0440adb10 con 0x7fe04c147500 2024-04-25T00:55:36.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.672+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe04c145160 con 0x7fe04c147500 2024-04-25T00:55:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.676+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04c142610 con 0x7fe04c147500 2024-04-25T00:55:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.676+0000 7fe033fff640 1 -- 172.21.15.112:0/1876346514 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe0440adcb0 con 0x7fe04c147500 2024-04-25T00:55:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.676+0000 7fe033fff640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe0380678c0 0x7fe038069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.676+0000 7fe033fff640 1 -- 172.21.15.112:0/1876346514 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe044120b10 con 0x7fe04c147500 2024-04-25T00:55:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.676+0000 7fe04affd640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe0380678c0 0x7fe038069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.676+0000 7fe04affd640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe0380678c0 0x7fe038069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe03c000f70 tx=0x7fe03c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.680+0000 7fe033fff640 1 -- 172.21.15.112:0/1876346514 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe04c144890 con 0x7fe04c147500 2024-04-25T00:55:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.840+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe04c068470 con 0x7fe04c147500 2024-04-25T00:55:36.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.840+0000 7fe033fff640 1 -- 172.21.15.112:0/1876346514 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7fe044016520 con 0x7fe04c147500 2024-04-25T00:55:36.844 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe0380678c0 msgr2=0x7fe038069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe0380678c0 0x7fe038069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe03c000f70 tx=0x7fe03c009040 comp rx=0 tx=0).stop 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c147500 msgr2=0x7fe04c143640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c147500 0x7fe04c143640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0440b0040 tx=0x7fe0440195b0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 shutdown_connections 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04c147500 0x7fe04c143640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe04c06e1d0 0x7fe04c1420d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe0380678c0 0x7fe038069d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 --2- 172.21.15.112:0/1876346514 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe04c06c620 0x7fe04c1dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 >> 172.21.15.112:0/1876346514 conn(0x7fe04c0711b0 msgr2=0x7fe04c145470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 shutdown_connections 2024-04-25T00:55:36.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.844+0000 7fe051819640 1 -- 172.21.15.112:0/1876346514 wait complete. 2024-04-25T00:55:36.863 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:55:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.972+0000 7f3088de2640 1 Processor -- start 2024-04-25T00:55:36.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.972+0000 7f3088de2640 1 -- start start 2024-04-25T00:55:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f3088de2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084145200 0x7f30840b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f3088de2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30840aeec0 con 0x7f30840b23f0 2024-04-25T00:55:36.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f3088de2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30840af780 con 0x7f30840add70 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f3088de2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30840b27d0 con 0x7f3084145200 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f30840b23f0 0x7f3084072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52532/0 (socket says 172.21.15.112:52532) 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 -- 172.21.15.112:0/827877723 learned_addr learned my addr 172.21.15.112:0/827877723 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f3082d9d640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084145200 0x7f30840b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 131347101 0 0) 0x7f30840aeec0 con 0x7f30840b23f0 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3070003530 con 0x7f30840b23f0 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 798135208 0 0) 0x7f3070003530 con 0x7f30840b23f0 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084145200 msgr2=0x7f30840b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084145200 0x7f30840b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 >> v1:172.21.15.187:6789/0 conn(0x7f30840add70 legacy=0x7f308414db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30840afa20 con 0x7f30840b23f0 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f307c001a20 con 0x7f30840b23f0 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 >> v1:172.21.15.112:6789/0 conn(0x7f30840b23f0 legacy=0x7f3084072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30700039f0 0x7f3070003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084145200 0x7f3070004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3070004d50 0x7f30700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30840b27d0 con 0x7f30700039f0 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f307000ba80 con 0x7f3070004d50 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f307000bc80 con 0x7f3084145200 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30700039f0 0x7f3070003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f3082d9d640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3070004d50 0x7f30700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 -- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084145200 msgr2=0x7f3070004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084145200 0x7f3070004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 -- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3070004d50 msgr2=0x7f30700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:36.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3070004d50 0x7f30700051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 -- 172.21.15.112:0/827877723 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3070003530 con 0x7f30700039f0 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308259c640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30700039f0 0x7f3070003e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30840b3610 tx=0x7f307c01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a916c6d589d759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c00a480 con 0x7f30700039f0 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c0ad630 con 0x7f30700039f0 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.976+0000 7f308159a640 1 -- 172.21.15.112:0/827877723 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0ada20 con 0x7f30700039f0 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- 172.21.15.112:0/827877723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30700039f0 msgr2=0x7f3070003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 --2- 172.21.15.112:0/827877723 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30700039f0 0x7f3070003e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30840b3610 tx=0x7f307c01d320 comp rx=0 tx=0).stop 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3082d9d640 1 -- 172.21.15.112:0/827877723 reap_dead start 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- 172.21.15.112:0/827877723 shutdown_connections 2024-04-25T00:55:36.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- 172.21.15.112:0/827877723 >> 172.21.15.112:0/827877723 conn(0x7f30840b5980 msgr2=0x7f30840b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:36.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- 172.21.15.112:0/827877723 shutdown_connections 2024-04-25T00:55:36.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- 172.21.15.112:0/827877723 wait complete. 2024-04-25T00:55:36.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 Processor -- start 2024-04-25T00:55:36.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- start start 2024-04-25T00:55:36.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f308413fd80 0x7f3084140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084140160 0x7f3084138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084138660 0x7f3084141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30840b52d0 con 0x7f308413fd80 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30840b5150 con 0x7f3084138660 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3084151a70 con 0x7f3084140160 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f308413fd80 0x7f3084140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f308413fd80 0x7f3084140990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38566/0 (socket says 172.21.15.112:38566) 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 -- 172.21.15.112:0/1576147517 learned_addr learned my addr 172.21.15.112:0/1576147517 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3081d9b640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084140160 0x7f3084138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3082d9d640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084138660 0x7f3084141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 -- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084140160 msgr2=0x7f3084138120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084140160 0x7f3084138120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 -- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084138660 msgr2=0x7f3084141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084138660 0x7f3084141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 -- 172.21.15.112:0/1576147517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3084141030 con 0x7f308413fd80 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3082d9d640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084138660 0x7f3084141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f308259c640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f308413fd80 0x7f3084140990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f307c01d360 tx=0x7f3070005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f307affd640 1 -- 172.21.15.112:0/1576147517 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0bb070 con 0x7f308413fd80 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f307affd640 1 -- 172.21.15.112:0/1576147517 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c0ad430 con 0x7f308413fd80 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3084141310 con 0x7f308413fd80 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f307affd640 1 -- 172.21.15.112:0/1576147517 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c0019e0 con 0x7f308413fd80 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3084142f20 con 0x7f308413fd80 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.980+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308404f580 con 0x7f308413fd80 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.984+0000 7f307affd640 1 -- 172.21.15.112:0/1576147517 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f307c0be070 con 0x7f308413fd80 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.984+0000 7f307affd640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30640678f0 0x7f3064069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.984+0000 7f307affd640 1 -- 172.21.15.112:0/1576147517 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f307c11fdf0 con 0x7f308413fd80 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.984+0000 7f3081d9b640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30640678f0 0x7f3064069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:36.988 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.984+0000 7f3081d9b640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30640678f0 0x7f3064069db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f308414e6e0 tx=0x7f306c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:36.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:36.988+0000 7f307affd640 1 -- 172.21.15.112:0/1576147517 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f307c0c5020 con 0x7f308413fd80 2024-04-25T00:55:37.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.136+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f308406b9c0 con 0x7f308413fd80 2024-04-25T00:55:37.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.136+0000 7f307affd640 1 -- 172.21.15.112:0/1576147517 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f307c017030 con 0x7f308413fd80 2024-04-25T00:55:37.140 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:37.140 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-04-25T00:55:37.140 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:55:37.140 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-04-25T00:55:37.140 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-04-25T00:55:37.140 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:55:37.140 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-04-25T00:55:37.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30640678f0 msgr2=0x7f3064069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30640678f0 0x7f3064069db0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f308414e6e0 tx=0x7f306c009040 comp rx=0 tx=0).stop 2024-04-25T00:55:37.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f308413fd80 msgr2=0x7f3084140990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f308413fd80 0x7f3084140990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f307c01d360 tx=0x7f3070005840 comp rx=0 tx=0).stop 2024-04-25T00:55:37.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 shutdown_connections 2024-04-25T00:55:37.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30640678f0 0x7f3064069db0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3084138660 0x7f3084141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3084140160 0x7f3084138120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 --2- 172.21.15.112:0/1576147517 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f308413fd80 0x7f3084140990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 >> 172.21.15.112:0/1576147517 conn(0x7f30840b5980 msgr2=0x7f308406ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:37.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 shutdown_connections 2024-04-25T00:55:37.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.140+0000 7f3088de2640 1 -- 172.21.15.112:0/1576147517 wait complete. 2024-04-25T00:55:37.157 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:55:37.157 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:55:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe2182c2640 1 Processor -- start 2024-04-25T00:55:37.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe2182c2640 1 -- start start 2024-04-25T00:55:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe2182c2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe2100b3510 0x7fe21013e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe2182c2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2100b38f0 con 0x7fe21013e6b0 2024-04-25T00:55:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe2182c2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2100b1530 con 0x7fe210147f30 2024-04-25T00:55:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe2182c2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe2100b12e0 con 0x7fe2100b3510 2024-04-25T00:55:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe21585d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe21013e6b0 0x7fe210147820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52536/0 (socket says 172.21.15.112:52536) 2024-04-25T00:55:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe21585d640 1 -- 172.21.15.112:0/3832275932 learned_addr learned my addr 172.21.15.112:0/3832275932 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:37.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe21605e640 1 --2- 172.21.15.112:0/3832275932 >> v2:172.21.15.112:3300/0 conn(0x7fe2100b3510 0x7fe21013e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe21505c640 1 -- 172.21.15.112:0/3832275932 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078911496 0 0) 0x7fe2100b38f0 con 0x7fe21013e6b0 2024-04-25T00:55:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe21505c640 1 -- 172.21.15.112:0/3832275932 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1fc003530 con 0x7fe21013e6b0 2024-04-25T00:55:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe21605e640 1 -- 172.21.15.112:0/3832275932 >> v1:172.21.15.187:6789/0 conn(0x7fe210147f30 legacy=0x7fe21014ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe21605e640 1 -- 172.21.15.112:0/3832275932 >> v1:172.21.15.112:6789/0 conn(0x7fe21013e6b0 legacy=0x7fe210147820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.264+0000 7fe21605e640 1 -- 172.21.15.112:0/3832275932 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2100b1810 con 0x7fe2100b3510 2024-04-25T00:55:37.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe21605e640 1 --2- 172.21.15.112:0/3832275932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe21013e170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0097c0 tx=0x7fe20c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c01d38387c4d37c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:37.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe21505c640 1 -- 172.21.15.112:0/3832275932 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0b1070 con 0x7fe2100b3510 2024-04-25T00:55:37.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe21505c640 1 -- 172.21.15.112:0/3832275932 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe20c0ac410 con 0x7fe2100b3510 2024-04-25T00:55:37.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe21505c640 1 -- 172.21.15.112:0/3832275932 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0ac7c0 con 0x7fe2100b3510 2024-04-25T00:55:37.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- 172.21.15.112:0/3832275932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 msgr2=0x7fe21013e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 --2- 172.21.15.112:0/3832275932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe21013e170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0097c0 tx=0x7fe20c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- 172.21.15.112:0/3832275932 shutdown_connections 2024-04-25T00:55:37.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 --2- 172.21.15.112:0/3832275932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe21013e170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- 172.21.15.112:0/3832275932 >> 172.21.15.112:0/3832275932 conn(0x7fe2100b8930 msgr2=0x7fe2100b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:37.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- 172.21.15.112:0/3832275932 shutdown_connections 2024-04-25T00:55:37.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- 172.21.15.112:0/3832275932 wait complete. 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 Processor -- start 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- start start 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe210143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe21013e6b0 0x7fe21013bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe210147f30 0x7fe21013c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe21006d910 con 0x7fe2100b3510 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe21006d790 con 0x7fe210147f30 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe2182c2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe210152990 con 0x7fe21013e6b0 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe21605e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe210143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe21605e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe210143c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38594/0 (socket says 172.21.15.112:38594) 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.268+0000 7fe21605e640 1 -- 172.21.15.112:0/3403214875 learned_addr learned my addr 172.21.15.112:0/3403214875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21605e640 1 -- 172.21.15.112:0/3403214875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe21013e6b0 msgr2=0x7fe21013bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21605e640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe21013e6b0 0x7fe21013bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21605e640 1 -- 172.21.15.112:0/3403214875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe210147f30 msgr2=0x7fe21013c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21685f640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe210147f30 0x7fe21013c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21605e640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe210147f30 0x7fe21013c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21605e640 1 -- 172.21.15.112:0/3403214875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2101420c0 con 0x7fe2100b3510 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21605e640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe210143c80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0097c0 tx=0x7fe20c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe206ffd640 1 -- 172.21.15.112:0/3403214875 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0b1040 con 0x7fe2100b3510 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe206ffd640 1 -- 172.21.15.112:0/3403214875 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe20c0ba070 con 0x7fe2100b3510 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21013baf0 con 0x7fe2100b3510 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe206ffd640 1 -- 172.21.15.112:0/3403214875 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0038a0 con 0x7fe2100b3510 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe210142640 con 0x7fe2100b3510 2024-04-25T00:55:37.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21013bcc0 con 0x7fe2100b3510 2024-04-25T00:55:37.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe206ffd640 1 -- 172.21.15.112:0/3403214875 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe20c003a40 con 0x7fe2100b3510 2024-04-25T00:55:37.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe206ffd640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe1f40678c0 0x7fe1f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21585d640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe1f40678c0 0x7fe1f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe206ffd640 1 -- 172.21.15.112:0/3403214875 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe20c11f460 con 0x7fe2100b3510 2024-04-25T00:55:37.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.272+0000 7fe21585d640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe1f40678c0 0x7fe1f4069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe200007fd0 tx=0x7fe200007940 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:37.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.276+0000 7fe206ffd640 1 -- 172.21.15.112:0/3403214875 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe20c016090 con 0x7fe2100b3510 2024-04-25T00:55:37.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.424+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe21013d710 con 0x7fe2100b3510 2024-04-25T00:55:37.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.572+0000 7fe206ffd640 1 -- 172.21.15.112:0/3403214875 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7fe21013d710 con 0x7fe2100b3510 2024-04-25T00:55:37.576 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-04-25T00:55:37.579 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe1f40678c0 msgr2=0x7fe1f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe1f40678c0 0x7fe1f4069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe200007fd0 tx=0x7fe200007940 comp rx=0 tx=0).stop 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 msgr2=0x7fe210143c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe210143c80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0097c0 tx=0x7fe20c019240 comp rx=0 tx=0).stop 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 shutdown_connections 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe1f40678c0 0x7fe1f4069d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe210147f30 0x7fe21013c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe21013e6b0 0x7fe21013bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 --2- 172.21.15.112:0/3403214875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2100b3510 0x7fe210143c80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 >> 172.21.15.112:0/3403214875 conn(0x7fe2100b8930 msgr2=0x7fe2100b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 shutdown_connections 2024-04-25T00:55:37.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.576+0000 7fe2182c2640 1 -- 172.21.15.112:0/3403214875 wait complete. 2024-04-25T00:55:37.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:55:37.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8822e57640 1 Processor -- start 2024-04-25T00:55:37.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8822e57640 1 -- start start 2024-04-25T00:55:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8822e57640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f881c0b9eb0 0x7f881c0b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8822e57640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f881c141010 0x7f881c141490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8822e57640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881c0b43f0 con 0x7f881c14a930 2024-04-25T00:55:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8822e57640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f881c0b4e60 con 0x7f881c0b9eb0 2024-04-25T00:55:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8822e57640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f881c0ba290 con 0x7f881c141010 2024-04-25T00:55:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8820bf3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f881c0b9eb0 0x7f881c0b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8820bf3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f881c0b9eb0 0x7f881c0b5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38602/0 (socket says 172.21.15.112:38602) 2024-04-25T00:55:37.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f88213f4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f881c14a930 0x7f881c0b7640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52552/0 (socket says 172.21.15.112:52552) 2024-04-25T00:55:37.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8820bf3640 1 -- 172.21.15.112:0/2937828875 learned_addr learned my addr 172.21.15.112:0/2937828875 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:37.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f881bfff640 1 --2- 172.21.15.112:0/2937828875 >> v2:172.21.15.187:3300/0 conn(0x7f881c141010 0x7f881c141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8820bf3640 1 -- 172.21.15.112:0/2937828875 >> v2:172.21.15.187:3300/0 conn(0x7f881c141010 msgr2=0x7f881c141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:37.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8820bf3640 1 --2- 172.21.15.112:0/2937828875 >> v2:172.21.15.187:3300/0 conn(0x7f881c141010 0x7f881c141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8820bf3640 1 -- 172.21.15.112:0/2937828875 >> v1:172.21.15.112:6789/0 conn(0x7f881c14a930 legacy=0x7f881c0b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f881b7fe640 1 -- 172.21.15.112:0/2937828875 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1558617940 0 0) 0x7f881c0b43f0 con 0x7f881c14a930 2024-04-25T00:55:37.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.728+0000 7f8820bf3640 1 -- 172.21.15.112:0/2937828875 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c0b5c70 con 0x7f881c0b9eb0 2024-04-25T00:55:37.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8820bf3640 1 --2- 172.21.15.112:0/2937828875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c0b9eb0 0x7f881c0b5730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f881c0b91b0 tx=0x7f881401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37a638e42561c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:37.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f881b7fe640 1 -- 172.21.15.112:0/2937828875 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140a5590 con 0x7f881c0b9eb0 2024-04-25T00:55:37.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f881b7fe640 1 -- 172.21.15.112:0/2937828875 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88140a5730 con 0x7f881c0b9eb0 2024-04-25T00:55:37.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f881b7fe640 1 -- 172.21.15.112:0/2937828875 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88140a5b00 con 0x7f881c0b9eb0 2024-04-25T00:55:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- 172.21.15.112:0/2937828875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c0b9eb0 msgr2=0x7f881c0b5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 --2- 172.21.15.112:0/2937828875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c0b9eb0 0x7f881c0b5730 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f881c0b91b0 tx=0x7f881401c8d0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- 172.21.15.112:0/2937828875 shutdown_connections 2024-04-25T00:55:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 --2- 172.21.15.112:0/2937828875 >> v2:172.21.15.187:3300/0 conn(0x7f881c141010 0x7f881c141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 --2- 172.21.15.112:0/2937828875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c0b9eb0 0x7f881c0b5730 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- 172.21.15.112:0/2937828875 >> 172.21.15.112:0/2937828875 conn(0x7f881c067350 msgr2=0x7f881c067760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- 172.21.15.112:0/2937828875 shutdown_connections 2024-04-25T00:55:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- 172.21.15.112:0/2937828875 wait complete. 2024-04-25T00:55:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 Processor -- start 2024-04-25T00:55:37.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- start start 2024-04-25T00:55:37.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f881c0b9eb0 0x7f881c13ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f881c141010 0x7f881c1e0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c14a930 0x7f881c147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f881c154e60 con 0x7f881c14a930 2024-04-25T00:55:37.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f881c154ce0 con 0x7f881c141010 2024-04-25T00:55:37.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8822e57640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f881c154fe0 con 0x7f881c0b9eb0 2024-04-25T00:55:37.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c14a930 0x7f881c147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c14a930 0x7f881c147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38616/0 (socket says 172.21.15.112:38616) 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 -- 172.21.15.112:0/2870301858 learned_addr learned my addr 172.21.15.112:0/2870301858 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f8820bf3640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f881c0b9eb0 0x7f881c13ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f881bfff640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f881c141010 0x7f881c1e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 -- 172.21.15.112:0/2870301858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f881c0b9eb0 msgr2=0x7f881c13ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f881c0b9eb0 0x7f881c13ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 -- 172.21.15.112:0/2870301858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f881c141010 msgr2=0x7f881c1e0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f881c141010 0x7f881c1e0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 -- 172.21.15.112:0/2870301858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c13e5b0 con 0x7f881c14a930 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88213f4640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c14a930 0x7f881c147ae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c3e0 tx=0x7f880c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.732+0000 7f88197fa640 1 -- 172.21.15.112:0/2870301858 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c0074e0 con 0x7f881c14a930 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f88197fa640 1 -- 172.21.15.112:0/2870301858 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880c0047f0 con 0x7f881c14a930 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c13e310 con 0x7f881c14a930 2024-04-25T00:55:37.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f88197fa640 1 -- 172.21.15.112:0/2870301858 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c004be0 con 0x7f881c14a930 2024-04-25T00:55:37.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c1e0350 con 0x7f881c14a930 2024-04-25T00:55:37.740 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c071830 con 0x7f881c14a930 2024-04-25T00:55:37.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f88197fa640 1 -- 172.21.15.112:0/2870301858 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f880c011070 con 0x7f881c14a930 2024-04-25T00:55:37.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f88197fa640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f87fc0678c0 0x7f87fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:37.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f88197fa640 1 -- 172.21.15.112:0/2870301858 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f880c0855b0 con 0x7f881c14a930 2024-04-25T00:55:37.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f8820bf3640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f87fc0678c0 0x7f87fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:37.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.736+0000 7f8820bf3640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f87fc0678c0 0x7f87fc069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88140ae000 tx=0x7f88140a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:37.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.740+0000 7f88197fa640 1 -- 172.21.15.112:0/2870301858 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f881c13e5b0 con 0x7f881c14a930 2024-04-25T00:55:37.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:37.900+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f881c068260 con 0x7f881c14a930 2024-04-25T00:55:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.576+0000 7f88197fa640 1 -- 172.21.15.112:0/2870301858 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7f880c056360 con 0x7f881c14a930 2024-04-25T00:55:38.580 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-04-25T00:55:38.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f87fc0678c0 msgr2=0x7f87fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f87fc0678c0 0x7f87fc069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88140ae000 tx=0x7f88140a7070 comp rx=0 tx=0).stop 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c14a930 msgr2=0x7f881c147ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c14a930 0x7f881c147ae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c3e0 tx=0x7f880c00c8a0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 shutdown_connections 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f881c14a930 0x7f881c147ae0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f87fc0678c0 0x7f87fc069d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f881c141010 0x7f881c1e0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 --2- 172.21.15.112:0/2870301858 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f881c0b9eb0 0x7f881c13ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 >> 172.21.15.112:0/2870301858 conn(0x7f881c067350 msgr2=0x7f881c0bd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 shutdown_connections 2024-04-25T00:55:38.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.580+0000 7f8822e57640 1 -- 172.21.15.112:0/2870301858 wait complete. 2024-04-25T00:55:38.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:55:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae3d12640 1 Processor -- start 2024-04-25T00:55:38.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae3d12640 1 -- start start 2024-04-25T00:55:38.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae3d12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc0b8e10 0x7f7adc09ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:38.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae3d12640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7adc09f380 0x7f7adc151f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:38.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae3d12640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7adc0bba90 con 0x7f7adc0723f0 2024-04-25T00:55:38.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae3d12640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7adc09f800 con 0x7f7adc09f380 2024-04-25T00:55:38.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae3d12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc06ea30 con 0x7f7adc0b8e10 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae12ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc0b8e10 0x7f7adc09ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae22af640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7adc09f380 0x7f7adc151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae22af640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7adc09f380 0x7f7adc151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:51762/0 (socket says 172.21.15.112:51762) 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae1aae640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f7adc0723f0 0x7f7adc1415b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:33284/0 (socket says 172.21.15.112:33284) 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae12ad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc0b8e10 0x7f7adc09ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:48778/0 (socket says 172.21.15.112:48778) 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae22af640 1 -- 172.21.15.112:0/2662511497 learned_addr learned my addr 172.21.15.112:0/2662511497 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae12ad640 1 -- 172.21.15.112:0/2662511497 >> v2:172.21.15.187:3300/0 conn(0x7f7adc09f380 msgr2=0x7f7adc151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae12ad640 1 --2- 172.21.15.112:0/2662511497 >> v2:172.21.15.187:3300/0 conn(0x7f7adc09f380 0x7f7adc151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae12ad640 1 -- 172.21.15.112:0/2662511497 >> v1:172.21.15.187:6789/0 conn(0x7f7adc0723f0 legacy=0x7f7adc1415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:38.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.752+0000 7f7ae12ad640 1 -- 172.21.15.112:0/2662511497 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc0a05e0 con 0x7f7adc0b8e10 2024-04-25T00:55:38.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae12ad640 1 --2- 172.21.15.112:0/2662511497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc0b8e10 0x7f7adc09ee40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0097c0 tx=0x7f7acc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b3805962914b9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:38.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae0aac640 1 -- 172.21.15.112:0/2662511497 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0aa600 con 0x7f7adc0b8e10 2024-04-25T00:55:38.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae0aac640 1 -- 172.21.15.112:0/2662511497 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7acc0af070 con 0x7f7adc0b8e10 2024-04-25T00:55:38.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae0aac640 1 -- 172.21.15.112:0/2662511497 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc0aa9c0 con 0x7f7adc0b8e10 2024-04-25T00:55:38.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2662511497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc0b8e10 msgr2=0x7f7adc09ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:38.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2662511497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc0b8e10 0x7f7adc09ee40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0097c0 tx=0x7f7acc01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2662511497 shutdown_connections 2024-04-25T00:55:38.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2662511497 >> v2:172.21.15.187:3300/0 conn(0x7f7adc09f380 0x7f7adc151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2662511497 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc0b8e10 0x7f7adc09ee40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2662511497 >> 172.21.15.112:0/2662511497 conn(0x7f7adc0b75d0 msgr2=0x7f7adc0b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:38.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2662511497 shutdown_connections 2024-04-25T00:55:38.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2662511497 wait complete. 2024-04-25T00:55:38.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 Processor -- start 2024-04-25T00:55:38.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- start start 2024-04-25T00:55:38.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7adc0723f0 0x7f7adc1e0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:38.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc09f380 0x7f7adc06bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:38.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7adc0b8e10 0x7f7adc06c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:38.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc0b9230 con 0x7f7adc0723f0 2024-04-25T00:55:38.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7adc06f620 con 0x7f7adc0b8e10 2024-04-25T00:55:38.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae3d12640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7adc141cc0 con 0x7f7adc09f380 2024-04-25T00:55:38.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae1aae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7adc0723f0 0x7f7adc1e0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae1aae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7adc0723f0 0x7f7adc1e0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38620/0 (socket says 172.21.15.112:38620) 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.756+0000 7f7ae1aae640 1 -- 172.21.15.112:0/2586827481 learned_addr learned my addr 172.21.15.112:0/2586827481 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae12ad640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc09f380 0x7f7adc06bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae22af640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7adc0b8e10 0x7f7adc06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae1aae640 1 -- 172.21.15.112:0/2586827481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc09f380 msgr2=0x7f7adc06bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae1aae640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc09f380 0x7f7adc06bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae1aae640 1 -- 172.21.15.112:0/2586827481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7adc0b8e10 msgr2=0x7f7adc06c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae1aae640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7adc0b8e10 0x7f7adc06c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae1aae640 1 -- 172.21.15.112:0/2586827481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7adc140b00 con 0x7f7adc0723f0 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae1aae640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7adc0723f0 0x7f7adc1e0080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c420 tx=0x7f7ad800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ad27fc640 1 -- 172.21.15.112:0/2586827481 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80074e0 con 0x7f7adc0723f0 2024-04-25T00:55:38.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ad27fc640 1 -- 172.21.15.112:0/2586827481 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad8007680 con 0x7f7adc0723f0 2024-04-25T00:55:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7adc140de0 con 0x7f7adc0723f0 2024-04-25T00:55:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ad27fc640 1 -- 172.21.15.112:0/2586827481 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004a30 con 0x7f7adc0723f0 2024-04-25T00:55:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7adc14b570 con 0x7f7adc0723f0 2024-04-25T00:55:38.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7adc0750a0 con 0x7f7adc0723f0 2024-04-25T00:55:38.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ad27fc640 1 -- 172.21.15.112:0/2586827481 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ad801a020 con 0x7f7adc0723f0 2024-04-25T00:55:38.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ad27fc640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7ac00678c0 0x7f7ac0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:38.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ad27fc640 1 -- 172.21.15.112:0/2586827481 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f7ad8085b10 con 0x7f7adc0723f0 2024-04-25T00:55:38.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae12ad640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7ac00678c0 0x7f7ac0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:38.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.760+0000 7f7ae12ad640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7ac00678c0 0x7f7ac0069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0b84b0 tx=0x7f7acc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:38.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.764+0000 7f7ad27fc640 1 -- 172.21.15.112:0/2586827481 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ad8021070 con 0x7f7adc0723f0 2024-04-25T00:55:38.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:55:38.853 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:55:38.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:38.960+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7adc068470 con 0x7f7adc0723f0 2024-04-25T00:55:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.616+0000 7f7ad27fc640 1 -- 172.21.15.112:0/2586827481 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f7ad8056820 con 0x7f7adc0723f0 2024-04-25T00:55:39.622 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:55:39.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.628+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7ac00678c0 msgr2=0x7f7ac0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:39.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.628+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7ac00678c0 0x7f7ac0069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0b84b0 tx=0x7f7acc0a7000 comp rx=0 tx=0).stop 2024-04-25T00:55:39.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.628+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7adc0723f0 msgr2=0x7f7adc1e0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:39.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.628+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7adc0723f0 0x7f7adc1e0080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c420 tx=0x7f7ad800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.632+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 shutdown_connections 2024-04-25T00:55:39.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.632+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7ac00678c0 0x7f7ac0069d80 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0b84b0 tx=0x7f7acc0a7000 comp rx=0 tx=0).stop 2024-04-25T00:55:39.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.632+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7adc0b8e10 0x7f7adc06c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.632+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7adc09f380 0x7f7adc06bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.632+0000 7f7ae3d12640 1 --2- 172.21.15.112:0/2586827481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7adc0723f0 0x7f7adc1e0080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.632+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 >> 172.21.15.112:0/2586827481 conn(0x7f7adc0b75d0 msgr2=0x7f7adc0b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:39.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.632+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 shutdown_connections 2024-04-25T00:55:39.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.632+0000 7f7ae3d12640 1 -- 172.21.15.112:0/2586827481 wait complete. 2024-04-25T00:55:39.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:39.786 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f794949f640 1 Processor -- start 2024-04-25T00:55:39.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f794949f640 1 -- start start 2024-04-25T00:55:39.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f794949f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79440b47f0 0x7f7944147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:39.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f794949f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f79440b2810 0x7f794414ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:39.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f794949f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79440b4bd0 con 0x7f7944148400 2024-04-25T00:55:39.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f794949f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f79440aff80 con 0x7f79440b2810 2024-04-25T00:55:39.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f794949f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f79440b1c70 con 0x7f79440b47f0 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f79437fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f79440b2810 0x7f794414ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f79427fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7944148400 0x7f79440b2100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52556/0 (socket says 172.21.15.112:52556) 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f79427fc640 1 -- 172.21.15.112:0/3115895895 learned_addr learned my addr 172.21.15.112:0/3115895895 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f7942ffd640 1 --2- 172.21.15.112:0/3115895895 >> v2:172.21.15.187:3300/0 conn(0x7f79440b47f0 0x7f7944147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f7941ffb640 1 -- 172.21.15.112:0/3115895895 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262975478 0 0) 0x7f79440b4bd0 con 0x7f7944148400 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f7941ffb640 1 -- 172.21.15.112:0/3115895895 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7930003530 con 0x7f7944148400 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f79437fe640 1 -- 172.21.15.112:0/3115895895 >> v2:172.21.15.187:3300/0 conn(0x7f79440b47f0 msgr2=0x7f7944147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f79437fe640 1 --2- 172.21.15.112:0/3115895895 >> v2:172.21.15.187:3300/0 conn(0x7f79440b47f0 0x7f7944147ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f79437fe640 1 -- 172.21.15.112:0/3115895895 >> v1:172.21.15.112:6789/0 conn(0x7f7944148400 legacy=0x7f79440b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:39.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.784+0000 7f79437fe640 1 -- 172.21.15.112:0/3115895895 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79440b1340 con 0x7f79440b2810 2024-04-25T00:55:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 --2- 172.21.15.112:0/3115895895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79440b2810 0x7f794414ebc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79440b5360 tx=0x7f793401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ba00af941ebeb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f7941ffb640 1 -- 172.21.15.112:0/3115895895 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340b2070 con 0x7f79440b2810 2024-04-25T00:55:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f7941ffb640 1 -- 172.21.15.112:0/3115895895 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79340a5760 con 0x7f79440b2810 2024-04-25T00:55:39.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f7941ffb640 1 -- 172.21.15.112:0/3115895895 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340a5b30 con 0x7f79440b2810 2024-04-25T00:55:39.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- 172.21.15.112:0/3115895895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79440b2810 msgr2=0x7f794414ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:39.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 --2- 172.21.15.112:0/3115895895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79440b2810 0x7f794414ebc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79440b5360 tx=0x7f793401c8a0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- 172.21.15.112:0/3115895895 shutdown_connections 2024-04-25T00:55:39.791 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 --2- 172.21.15.112:0/3115895895 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f79440b2810 0x7f794414ebc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 --2- 172.21.15.112:0/3115895895 >> v2:172.21.15.187:3300/0 conn(0x7f79440b47f0 0x7f7944147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- 172.21.15.112:0/3115895895 >> 172.21.15.112:0/3115895895 conn(0x7f79440b5980 msgr2=0x7f79440b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:39.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- 172.21.15.112:0/3115895895 shutdown_connections 2024-04-25T00:55:39.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- 172.21.15.112:0/3115895895 wait complete. 2024-04-25T00:55:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 Processor -- start 2024-04-25T00:55:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- start start 2024-04-25T00:55:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79440b2810 0x7f794409e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79440b47f0 0x7f79440b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7944148400 0x7f79441dc020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79440ae280 con 0x7f7944148400 2024-04-25T00:55:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79440b1060 con 0x7f79440b47f0 2024-04-25T00:55:39.794 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f794949f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f79440ae400 con 0x7f79440b2810 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7944148400 0x7f79441dc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7944148400 0x7f79441dc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38634/0 (socket says 172.21.15.112:38634) 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 -- 172.21.15.112:0/1385631912 learned_addr learned my addr 172.21.15.112:0/1385631912 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f7942ffd640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79440b2810 0x7f794409e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79427fc640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79440b47f0 0x7f79440b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 -- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79440b2810 msgr2=0x7f794409e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79440b2810 0x7f794409e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 -- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79440b47f0 msgr2=0x7f79440b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79440b47f0 0x7f79440b87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 -- 172.21.15.112:0/1385631912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79441db890 con 0x7f7944148400 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f7942ffd640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79440b2810 0x7f794409e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f79437fe640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7944148400 0x7f79441dc020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79340b0040 tx=0x7f7934019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f793b7fe640 1 -- 172.21.15.112:0/1385631912 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79340b2050 con 0x7f7944148400 2024-04-25T00:55:39.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f793b7fe640 1 -- 172.21.15.112:0/1385631912 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79340be070 con 0x7f7944148400 2024-04-25T00:55:39.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.788+0000 7f793b7fe640 1 -- 172.21.15.112:0/1385631912 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934009450 con 0x7f7944148400 2024-04-25T00:55:39.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.792+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794409ec70 con 0x7f7944148400 2024-04-25T00:55:39.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.792+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f794409aa30 con 0x7f7944148400 2024-04-25T00:55:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.792+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f794404f580 con 0x7f7944148400 2024-04-25T00:55:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.792+0000 7f793b7fe640 1 -- 172.21.15.112:0/1385631912 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f79340ad870 con 0x7f7944148400 2024-04-25T00:55:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.792+0000 7f793b7fe640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f79240678c0 0x7f7924069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.792+0000 7f793b7fe640 1 -- 172.21.15.112:0/1385631912 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f7934120b60 con 0x7f7944148400 2024-04-25T00:55:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.792+0000 7f7942ffd640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f79240678c0 0x7f7924069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:39.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.792+0000 7f7942ffd640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f79240678c0 0x7f7924069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f79440af3e0 tx=0x7f793c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:39.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.796+0000 7f793b7fe640 1 -- 172.21.15.112:0/1385631912 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7934016520 con 0x7f7944148400 2024-04-25T00:55:39.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.956+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f794406cea0 con 0x7f7944148400 2024-04-25T00:55:39.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f793b7fe640 1 -- 172.21.15.112:0/1385631912 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16666 (secure 0 0 0) 0x7f79340f1730 con 0x7f7944148400 2024-04-25T00:55:39.963 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:39.963 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:38.966173+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:39.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f79240678c0 msgr2=0x7f7924069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:39.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f79240678c0 0x7f7924069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f79440af3e0 tx=0x7f793c008040 comp rx=0 tx=0).stop 2024-04-25T00:55:39.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7944148400 msgr2=0x7f79441dc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:39.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7944148400 0x7f79441dc020 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79340b0040 tx=0x7f7934019640 comp rx=0 tx=0).stop 2024-04-25T00:55:39.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 shutdown_connections 2024-04-25T00:55:39.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7944148400 0x7f79441dc020 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f79240678c0 0x7f7924069d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f79440b47f0 0x7f79440b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 --2- 172.21.15.112:0/1385631912 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79440b2810 0x7f794409e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:39.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 >> 172.21.15.112:0/1385631912 conn(0x7f79440b5980 msgr2=0x7f7944069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:39.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 shutdown_connections 2024-04-25T00:55:39.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:39.960+0000 7f794949f640 1 -- 172.21.15.112:0/1385631912 wait complete. 2024-04-25T00:55:39.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:40.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.096+0000 7f00f2cb9640 1 Processor -- start 2024-04-25T00:55:40.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.096+0000 7f00f2cb9640 1 -- start start 2024-04-25T00:55:40.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f2cb9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec0ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f2cb9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00ec0aee00 0x7f00ec0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f2cb9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ec0b52f0 con 0x7f00ec0ae9a0 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f2cb9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f00ec0b0800 con 0x7f00ec0aee00 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f2cb9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec143f50 con 0x7f00ec0adfd0 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f0a55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec0ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f0a55640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec0ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:48796/0 (socket says 172.21.15.112:48796) 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f0a55640 1 -- 172.21.15.112:0/3320428262 learned_addr learned my addr 172.21.15.112:0/3320428262 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f1256640 1 --2- 172.21.15.112:0/3320428262 >> v2:172.21.15.187:3300/0 conn(0x7f00ec0aee00 0x7f00ec0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f0a55640 1 -- 172.21.15.112:0/3320428262 >> v2:172.21.15.187:3300/0 conn(0x7f00ec0aee00 msgr2=0x7f00ec0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f0a55640 1 --2- 172.21.15.112:0/3320428262 >> v2:172.21.15.187:3300/0 conn(0x7f00ec0aee00 0x7f00ec0b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f0a55640 1 -- 172.21.15.112:0/3320428262 >> v1:172.21.15.187:6789/0 conn(0x7f00ec0ae9a0 legacy=0x7f00ec0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.100+0000 7f00f0a55640 1 -- 172.21.15.112:0/3320428262 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec144230 con 0x7f00ec0adfd0 2024-04-25T00:55:40.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f0a55640 1 --2- 172.21.15.112:0/3320428262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec0ae3b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009870 tx=0x7f00e401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c1a9bb31d77cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:40.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00eb7fe640 1 -- 172.21.15.112:0/3320428262 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40ab5f0 con 0x7f00ec0adfd0 2024-04-25T00:55:40.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00eb7fe640 1 -- 172.21.15.112:0/3320428262 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e40b0070 con 0x7f00ec0adfd0 2024-04-25T00:55:40.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00eb7fe640 1 -- 172.21.15.112:0/3320428262 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40ab9b0 con 0x7f00ec0adfd0 2024-04-25T00:55:40.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3320428262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 msgr2=0x7f00ec0ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3320428262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec0ae3b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009870 tx=0x7f00e401c4d0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3320428262 shutdown_connections 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3320428262 >> v2:172.21.15.187:3300/0 conn(0x7f00ec0aee00 0x7f00ec0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3320428262 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec0ae3b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3320428262 >> 172.21.15.112:0/3320428262 conn(0x7f00ec0b5980 msgr2=0x7f00ec0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3320428262 shutdown_connections 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3320428262 wait complete. 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 Processor -- start 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- start start 2024-04-25T00:55:40.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec132d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00ec0ae9a0 0x7f00ec070340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec0aee00 0x7f00ec070880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec14e350 con 0x7f00ec0aee00 2024-04-25T00:55:40.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec14e1d0 con 0x7f00ec0ae9a0 2024-04-25T00:55:40.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f2cb9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec14e4d0 con 0x7f00ec0adfd0 2024-04-25T00:55:40.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f1256640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec0aee00 0x7f00ec070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f1256640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec0aee00 0x7f00ec070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38650/0 (socket says 172.21.15.112:38650) 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f1256640 1 -- 172.21.15.112:0/3579297630 learned_addr learned my addr 172.21.15.112:0/3579297630 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f0a55640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00ebfff640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00ec0ae9a0 0x7f00ec070340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f1256640 1 -- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 msgr2=0x7f00ec132d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f1256640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec132d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f1256640 1 -- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00ec0ae9a0 msgr2=0x7f00ec070340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f1256640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00ec0ae9a0 0x7f00ec070340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00f1256640 1 -- 172.21.15.112:0/3579297630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec133770 con 0x7f00ec0aee00 2024-04-25T00:55:40.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.104+0000 7f00ebfff640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00ec0ae9a0 0x7f00ec070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00f1256640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec0aee00 0x7f00ec070880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e820 tx=0x7f00dc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00e97fa640 1 -- 172.21.15.112:0/3579297630 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc017070 con 0x7f00ec0aee00 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00e97fa640 1 -- 172.21.15.112:0/3579297630 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc00c730 con 0x7f00ec0aee00 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec072220 con 0x7f00ec0aee00 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00e97fa640 1 -- 172.21.15.112:0/3579297630 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00cae0 con 0x7f00ec0aee00 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec13e5f0 con 0x7f00ec0aee00 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00ec04f580 con 0x7f00ec0aee00 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00e97fa640 1 -- 172.21.15.112:0/3579297630 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00dc01a070 con 0x7f00ec0aee00 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00e97fa640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00cc0678f0 0x7f00cc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00f0a55640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00cc0678f0 0x7f00cc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00e97fa640 1 -- 172.21.15.112:0/3579297630 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f00dc0861c0 con 0x7f00ec0aee00 2024-04-25T00:55:40.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.108+0000 7f00f0a55640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00cc0678f0 0x7f00cc069db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009870 tx=0x7f00e40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:40.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.112+0000 7f00e97fa640 1 -- 172.21.15.112:0/3579297630 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f00dc056690 con 0x7f00ec0aee00 2024-04-25T00:55:40.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.272+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00ec0b6e60 con 0x7f00ec0aee00 2024-04-25T00:55:40.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.276+0000 7f00e97fa640 1 -- 172.21.15.112:0/3579297630 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16666 (secure 0 0 0) 0x7f00dc05a990 con 0x7f00ec0aee00 2024-04-25T00:55:40.280 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:40.280 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":19,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:38.966173+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:40.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.276+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00cc0678f0 msgr2=0x7f00cc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.276+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00cc0678f0 0x7f00cc069db0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009870 tx=0x7f00e40a7000 comp rx=0 tx=0).stop 2024-04-25T00:55:40.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec0aee00 msgr2=0x7f00ec070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec0aee00 0x7f00ec070880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e820 tx=0x7f00dc00ece0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 shutdown_connections 2024-04-25T00:55:40.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00ec0aee00 0x7f00ec070880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00cc0678f0 0x7f00cc069db0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00ec0ae9a0 0x7f00ec070340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 --2- 172.21.15.112:0/3579297630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00ec0adfd0 0x7f00ec132d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 >> 172.21.15.112:0/3579297630 conn(0x7f00ec0b5980 msgr2=0x7f00ec0b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 shutdown_connections 2024-04-25T00:55:40.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.280+0000 7f00f2cb9640 1 -- 172.21.15.112:0/3579297630 wait complete. 2024-04-25T00:55:40.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:55:40.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39de823640 1 Processor -- start 2024-04-25T00:55:40.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39de823640 1 -- start start 2024-04-25T00:55:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39de823640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d80affb0 0x7f39d809da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39de823640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f39d80b0390 0x7f39d814fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39de823640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d80b1390 con 0x7f39d809dfa0 2024-04-25T00:55:40.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39de823640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f39d80b1d70 con 0x7f39d80b0390 2024-04-25T00:55:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39de823640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39d80b1600 con 0x7f39d80affb0 2024-04-25T00:55:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39dcdc0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f39d80b0390 0x7f39d814fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39d7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d80affb0 0x7f39d809da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39d7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d80affb0 0x7f39d809da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:48804/0 (socket says 172.21.15.112:48804) 2024-04-25T00:55:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39d7fff640 1 -- 172.21.15.112:0/2404666642 learned_addr learned my addr 172.21.15.112:0/2404666642 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39d7fff640 1 -- 172.21.15.112:0/2404666642 >> v2:172.21.15.187:3300/0 conn(0x7f39d80b0390 msgr2=0x7f39d814fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39d7fff640 1 --2- 172.21.15.112:0/2404666642 >> v2:172.21.15.187:3300/0 conn(0x7f39d80b0390 0x7f39d814fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39d7fff640 1 -- 172.21.15.112:0/2404666642 >> v1:172.21.15.187:6789/0 conn(0x7f39d809dfa0 legacy=0x7f39d8148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.421 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.416+0000 7f39d7fff640 1 -- 172.21.15.112:0/2404666642 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d80b1fa0 con 0x7f39d80affb0 2024-04-25T00:55:40.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39d7fff640 1 --2- 172.21.15.112:0/2404666642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d80affb0 0x7f39d809da60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009870 tx=0x7f39cc01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=577f8df810865209 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:40.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39d6ffd640 1 -- 172.21.15.112:0/2404666642 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0aa430 con 0x7f39d80affb0 2024-04-25T00:55:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39d6ffd640 1 -- 172.21.15.112:0/2404666642 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39cc0af070 con 0x7f39d80affb0 2024-04-25T00:55:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39d6ffd640 1 -- 172.21.15.112:0/2404666642 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0aa7f0 con 0x7f39d80affb0 2024-04-25T00:55:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- 172.21.15.112:0/2404666642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d80affb0 msgr2=0x7f39d809da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 --2- 172.21.15.112:0/2404666642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d80affb0 0x7f39d809da60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009870 tx=0x7f39cc01c4d0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- 172.21.15.112:0/2404666642 shutdown_connections 2024-04-25T00:55:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 --2- 172.21.15.112:0/2404666642 >> v2:172.21.15.187:3300/0 conn(0x7f39d80b0390 0x7f39d814fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 --2- 172.21.15.112:0/2404666642 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d80affb0 0x7f39d809da60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- 172.21.15.112:0/2404666642 >> 172.21.15.112:0/2404666642 conn(0x7f39d80b8c80 msgr2=0x7f39d80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- 172.21.15.112:0/2404666642 shutdown_connections 2024-04-25T00:55:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- 172.21.15.112:0/2404666642 wait complete. 2024-04-25T00:55:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 Processor -- start 2024-04-25T00:55:40.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- start start 2024-04-25T00:55:40.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d809dfa0 0x7f39d8138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d80affb0 0x7f39d813d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d80b0390 0x7f39d81e41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8152950 con 0x7f39d80affb0 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81527d0 con 0x7f39d80b0390 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39de823640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f39d8152ad0 con 0x7f39d809dfa0 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d80affb0 0x7f39d813d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39d77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d80affb0 0x7f39d813d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38666/0 (socket says 172.21.15.112:38666) 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39d77fe640 1 -- 172.21.15.112:0/3292613536 learned_addr learned my addr 172.21.15.112:0/3292613536 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39d7fff640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d809dfa0 0x7f39d8138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.420+0000 7f39dcdc0640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d80b0390 0x7f39d81e41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d77fe640 1 -- 172.21.15.112:0/3292613536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d809dfa0 msgr2=0x7f39d8138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d77fe640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d809dfa0 0x7f39d8138c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d77fe640 1 -- 172.21.15.112:0/3292613536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d80b0390 msgr2=0x7f39d81e41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:40.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d77fe640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d80b0390 0x7f39d81e41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d77fe640 1 -- 172.21.15.112:0/3292613536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d814de00 con 0x7f39d80affb0 2024-04-25T00:55:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d77fe640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d80affb0 0x7f39d813d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c370 tx=0x7f39c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d4ff9640 1 -- 172.21.15.112:0/3292613536 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80052d0 con 0x7f39d80affb0 2024-04-25T00:55:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d4ff9640 1 -- 172.21.15.112:0/3292613536 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c8005470 con 0x7f39d80affb0 2024-04-25T00:55:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d813d920 con 0x7f39d80affb0 2024-04-25T00:55:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d4ff9640 1 -- 172.21.15.112:0/3292613536 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8007260 con 0x7f39d80affb0 2024-04-25T00:55:40.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d813dd10 con 0x7f39d80affb0 2024-04-25T00:55:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d814fd90 con 0x7f39d80affb0 2024-04-25T00:55:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d4ff9640 1 -- 172.21.15.112:0/3292613536 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f39c8007400 con 0x7f39d80affb0 2024-04-25T00:55:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d4ff9640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f39c40678f0 0x7f39c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d4ff9640 1 -- 172.21.15.112:0/3292613536 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f39c8084fa0 con 0x7f39d80affb0 2024-04-25T00:55:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d7fff640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f39c40678f0 0x7f39c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:40.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.424+0000 7f39d7fff640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f39c40678f0 0x7f39c4069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009870 tx=0x7f39cc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:40.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.428+0000 7f39d4ff9640 1 -- 172.21.15.112:0/3292613536 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f39d814de00 con 0x7f39d80affb0 2024-04-25T00:55:40.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:40.628+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f39d80b1040 con 0x7f39d80affb0 2024-04-25T00:55:41.632 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.628+0000 7f39d4ff9640 1 -- 172.21.15.112:0/3292613536 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f39c8055520 con 0x7f39d80affb0 2024-04-25T00:55:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f39c40678f0 msgr2=0x7f39c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f39c40678f0 0x7f39c4069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009870 tx=0x7f39cc0a7000 comp rx=0 tx=0).stop 2024-04-25T00:55:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d80affb0 msgr2=0x7f39d813d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d80affb0 0x7f39d813d0c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c370 tx=0x7f39c800c830 comp rx=0 tx=0).stop 2024-04-25T00:55:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 shutdown_connections 2024-04-25T00:55:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f39c40678f0 0x7f39c4069db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f39d80b0390 0x7f39d81e41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f39d80affb0 0x7f39d813d0c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 --2- 172.21.15.112:0/3292613536 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f39d809dfa0 0x7f39d8138c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 >> 172.21.15.112:0/3292613536 conn(0x7f39d80b8c80 msgr2=0x7f39d80b7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:41.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 shutdown_connections 2024-04-25T00:55:41.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.632+0000 7f39de823640 1 -- 172.21.15.112:0/3292613536 wait complete. 2024-04-25T00:55:41.659 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:55:41.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:55:41.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9228d35640 1 Processor -- start 2024-04-25T00:55:41.795 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9228d35640 1 -- start start 2024-04-25T00:55:41.796 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9228d35640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922406c170 0x7f92240ade80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9228d35640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f92240aeda0 0x7f922406c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9228d35640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92240af2f0 con 0x7f922406ca90 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9228d35640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f922406d5f0 con 0x7f92240aeda0 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9228d35640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f92240b4d90 con 0x7f922406c170 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9219d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f92240aeda0 0x7f922406c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9222d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f922406ca90 0x7f922406cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:52560/0 (socket says 172.21.15.112:52560) 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9222d9d640 1 -- 172.21.15.112:0/3437905579 learned_addr learned my addr 172.21.15.112:0/3437905579 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f922259c640 1 --2- 172.21.15.112:0/3437905579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922406c170 0x7f92240ade80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9219d9b640 1 -- 172.21.15.112:0/3437905579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922406c170 msgr2=0x7f92240ade80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9219d9b640 1 --2- 172.21.15.112:0/3437905579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922406c170 0x7f92240ade80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9219d9b640 1 -- 172.21.15.112:0/3437905579 >> v1:172.21.15.112:6789/0 conn(0x7f922406ca90 legacy=0x7f922406cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:41.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.792+0000 7f9219d9b640 1 -- 172.21.15.112:0/3437905579 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92240af5d0 con 0x7f92240aeda0 2024-04-25T00:55:41.799 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9219d9b640 1 --2- 172.21.15.112:0/3437905579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92240aeda0 0x7f922406c550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f920c0097c0 tx=0x7f920c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685a28594cb6e802 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9221d9b640 1 -- 172.21.15.112:0/3437905579 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b0070 con 0x7f92240aeda0 2024-04-25T00:55:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9221d9b640 1 -- 172.21.15.112:0/3437905579 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920c0ab600 con 0x7f92240aeda0 2024-04-25T00:55:41.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9221d9b640 1 -- 172.21.15.112:0/3437905579 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c0ab9d0 con 0x7f92240aeda0 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- 172.21.15.112:0/3437905579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92240aeda0 msgr2=0x7f922406c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 --2- 172.21.15.112:0/3437905579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92240aeda0 0x7f922406c550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f920c0097c0 tx=0x7f920c01c380 comp rx=0 tx=0).stop 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- 172.21.15.112:0/3437905579 shutdown_connections 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 --2- 172.21.15.112:0/3437905579 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f92240aeda0 0x7f922406c550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 --2- 172.21.15.112:0/3437905579 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f922406c170 0x7f92240ade80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- 172.21.15.112:0/3437905579 >> 172.21.15.112:0/3437905579 conn(0x7f92240b8c80 msgr2=0x7f92240b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- 172.21.15.112:0/3437905579 shutdown_connections 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- 172.21.15.112:0/3437905579 wait complete. 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 Processor -- start 2024-04-25T00:55:41.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- start start 2024-04-25T00:55:41.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406c170 0x7f922414f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f922406ca90 0x7f92241e7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92240aeda0 0x7f9224146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f92240b2e60 con 0x7f922406ca90 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f92240ae3c0 con 0x7f922406c170 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9228d35640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f922409ad80 con 0x7f92240aeda0 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9219d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f922406ca90 0x7f92241e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9219d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f922406ca90 0x7f92241e7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38688/0 (socket says 172.21.15.112:38688) 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9219d9b640 1 -- 172.21.15.112:0/922518805 learned_addr learned my addr 172.21.15.112:0/922518805 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9222d9d640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92240aeda0 0x7f9224146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.796+0000 7f9219d9b640 1 -- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92240aeda0 msgr2=0x7f9224146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f922259c640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406c170 0x7f922414f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:41.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9219d9b640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92240aeda0 0x7f9224146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9219d9b640 1 -- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406c170 msgr2=0x7f922414f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:41.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9219d9b640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406c170 0x7f922414f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:41.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9219d9b640 1 -- 172.21.15.112:0/922518805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922409aff0 con 0x7f922406ca90 2024-04-25T00:55:41.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9222d9d640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92240aeda0 0x7f9224146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:41.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9219d9b640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f922406ca90 0x7f92241e7390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ae040 tx=0x7f920c019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:41.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f921b7fe640 1 -- 172.21.15.112:0/922518805 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b0040 con 0x7f922406ca90 2024-04-25T00:55:41.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f921b7fe640 1 -- 172.21.15.112:0/922518805 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920c0ab400 con 0x7f922406ca90 2024-04-25T00:55:41.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9224151120 con 0x7f922406ca90 2024-04-25T00:55:41.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f921b7fe640 1 -- 172.21.15.112:0/922518805 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b4630 con 0x7f922406ca90 2024-04-25T00:55:41.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922414dde0 con 0x7f922406ca90 2024-04-25T00:55:41.806 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922406c870 con 0x7f922406ca90 2024-04-25T00:55:41.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f921b7fe640 1 -- 172.21.15.112:0/922518805 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f920c0bb030 con 0x7f922406ca90 2024-04-25T00:55:41.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f921b7fe640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f92040678c0 0x7f9204069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:41.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f922259c640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f92040678c0 0x7f9204069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:41.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f921b7fe640 1 -- 172.21.15.112:0/922518805 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f920c11fa70 con 0x7f922406ca90 2024-04-25T00:55:41.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.800+0000 7f922259c640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f92040678c0 0x7f9204069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f92140096f0 tx=0x7f9214008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:41.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.804+0000 7f921b7fe640 1 -- 172.21.15.112:0/922518805 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f920c0c2060 con 0x7f922406ca90 2024-04-25T00:55:42.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:41.996+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f922404f5f0 con 0x7f922406ca90 2024-04-25T00:55:42.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f921b7fe640 1 -- 172.21.15.112:0/922518805 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f920c016090 con 0x7f922406ca90 2024-04-25T00:55:42.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f92040678c0 msgr2=0x7f9204069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f92040678c0 0x7f9204069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f92140096f0 tx=0x7f9214008040 comp rx=0 tx=0).stop 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f922406ca90 msgr2=0x7f92241e7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f922406ca90 0x7f92241e7390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f920c0ae040 tx=0x7f920c019120 comp rx=0 tx=0).stop 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 shutdown_connections 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f92240aeda0 0x7f9224146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f922406ca90 0x7f92241e7390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f92040678c0 0x7f9204069d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.640+0000 7f9228d35640 1 --2- 172.21.15.112:0/922518805 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f922406c170 0x7f922414f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.644+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 >> 172.21.15.112:0/922518805 conn(0x7f92240b8c80 msgr2=0x7f92240b9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.644+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 shutdown_connections 2024-04-25T00:55:42.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.644+0000 7f9228d35640 1 -- 172.21.15.112:0/922518805 wait complete. 2024-04-25T00:55:42.669 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:55:42.669 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:55:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.792+0000 7fc33a18d640 1 Processor -- start 2024-04-25T00:55:42.797 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.792+0000 7fc33a18d640 1 -- start start 2024-04-25T00:55:42.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.792+0000 7fc33a18d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc33406c520 0x7fc3340727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:42.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3340b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:42.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3340b5300 con 0x7fc334141110 2024-04-25T00:55:42.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc33406c900 con 0x7fc33406c520 2024-04-25T00:55:42.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc3340ae670 con 0x7fc3340b47d0 2024-04-25T00:55:42.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3337fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc33406c520 0x7fc3340727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:42.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3337fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc33406c520 0x7fc3340727d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38696/0 (socket says 172.21.15.112:38696) 2024-04-25T00:55:42.800 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3337fe640 1 -- 172.21.15.112:0/1291562820 learned_addr learned my addr 172.21.15.112:0/1291562820 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:42.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc332ffd640 1 --2- 172.21.15.112:0/1291562820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3340b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:42.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3337fe640 1 -- 172.21.15.112:0/1291562820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 msgr2=0x7fc3340b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:42.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3337fe640 1 --2- 172.21.15.112:0/1291562820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3340b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3337fe640 1 -- 172.21.15.112:0/1291562820 >> v1:172.21.15.187:6789/0 conn(0x7fc334141110 legacy=0x7fc3341415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:42.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3337fe640 1 -- 172.21.15.112:0/1291562820 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3340ae950 con 0x7fc33406c520 2024-04-25T00:55:42.801 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc332ffd640 1 --2- 172.21.15.112:0/1291562820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3340b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:42.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3337fe640 1 --2- 172.21.15.112:0/1291562820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc3340727d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009870 tx=0x7fc31c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef77fa9d9df0d1cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:42.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3327fc640 1 -- 172.21.15.112:0/1291562820 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0b0070 con 0x7fc33406c520 2024-04-25T00:55:42.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3327fc640 1 -- 172.21.15.112:0/1291562820 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c0ab600 con 0x7fc33406c520 2024-04-25T00:55:42.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc3327fc640 1 -- 172.21.15.112:0/1291562820 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0ab9d0 con 0x7fc33406c520 2024-04-25T00:55:42.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 -- 172.21.15.112:0/1291562820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 msgr2=0x7fc3340727d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:42.803 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 --2- 172.21.15.112:0/1291562820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc3340727d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009870 tx=0x7fc31c01c490 comp rx=0 tx=0).stop 2024-04-25T00:55:42.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 -- 172.21.15.112:0/1291562820 shutdown_connections 2024-04-25T00:55:42.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 --2- 172.21.15.112:0/1291562820 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3340b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 --2- 172.21.15.112:0/1291562820 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc3340727d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 -- 172.21.15.112:0/1291562820 >> 172.21.15.112:0/1291562820 conn(0x7fc3340b5980 msgr2=0x7fc3340b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:42.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 -- 172.21.15.112:0/1291562820 shutdown_connections 2024-04-25T00:55:42.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.796+0000 7fc33a18d640 1 -- 172.21.15.112:0/1291562820 wait complete. 2024-04-25T00:55:42.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 Processor -- start 2024-04-25T00:55:42.804 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 -- start start 2024-04-25T00:55:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc33413c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3341e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc334141110 0x7fc3341e0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc3341548c0 con 0x7fc33406c520 2024-04-25T00:55:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc334154740 con 0x7fc334141110 2024-04-25T00:55:42.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc334154a40 con 0x7fc3340b47d0 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc33413c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc33413c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:38710/0 (socket says 172.21.15.112:38710) 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 -- 172.21.15.112:0/2637106504 learned_addr learned my addr 172.21.15.112:0/2637106504 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc332ffd640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3341e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 -- 172.21.15.112:0/2637106504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 msgr2=0x7fc3341e01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3341e01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 -- 172.21.15.112:0/2637106504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc334141110 msgr2=0x7fc3341e0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc333fff640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc334141110 0x7fc3341e0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc334141110 0x7fc3341e0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:42.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 -- 172.21.15.112:0/2637106504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33413a730 con 0x7fc33406c520 2024-04-25T00:55:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc3337fe640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc33413c130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0ae040 tx=0x7fc31c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc32bfff640 1 -- 172.21.15.112:0/2637106504 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0ba020 con 0x7fc33406c520 2024-04-25T00:55:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc32bfff640 1 -- 172.21.15.112:0/2637106504 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c0b6640 con 0x7fc33406c520 2024-04-25T00:55:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33413ca90 con 0x7fc33406c520 2024-04-25T00:55:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc32bfff640 1 -- 172.21.15.112:0/2637106504 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31c0b0040 con 0x7fc33406c520 2024-04-25T00:55:42.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3340b2450 con 0x7fc33406c520 2024-04-25T00:55:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33404f580 con 0x7fc33406c520 2024-04-25T00:55:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.800+0000 7fc32bfff640 1 -- 172.21.15.112:0/2637106504 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc31c0bd020 con 0x7fc33406c520 2024-04-25T00:55:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.804+0000 7fc32bfff640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc3140678c0 0x7fc314069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.804+0000 7fc32bfff640 1 -- 172.21.15.112:0/2637106504 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc31c11fe50 con 0x7fc33406c520 2024-04-25T00:55:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.804+0000 7fc332ffd640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc3140678c0 0x7fc314069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:42.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.804+0000 7fc332ffd640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc3140678c0 0x7fc314069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3341dda30 tx=0x7fc3240084d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:42.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.808+0000 7fc32bfff640 1 -- 172.21.15.112:0/2637106504 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc31c0c4030 con 0x7fc33406c520 2024-04-25T00:55:42.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.992+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc33413c840 con 0x7fc33406c520 2024-04-25T00:55:42.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.996+0000 7fc32bfff640 1 -- 172.21.15.112:0/2637106504 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7fc31c016110 con 0x7fc33406c520 2024-04-25T00:55:42.999 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:42.999 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:55:38.966915+0000","modified":"2024-04-25T00:55:42.002877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:55:42.999 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T00:55:43.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.996+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc3140678c0 msgr2=0x7fc314069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:42.996+0000 7fc33a18d640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc3140678c0 0x7fc314069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3341dda30 tx=0x7fc3240084d0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 msgr2=0x7fc33413c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc33413c130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc31c0ae040 tx=0x7fc31c01acd0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 shutdown_connections 2024-04-25T00:55:43.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc3140678c0 0x7fc314069d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc334141110 0x7fc3341e0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc3340b47d0 0x7fc3341e01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 --2- 172.21.15.112:0/2637106504 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc33406c520 0x7fc33413c130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 >> 172.21.15.112:0/2637106504 conn(0x7fc3340b5980 msgr2=0x7fc334145c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 shutdown_connections 2024-04-25T00:55:43.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.000+0000 7fc33a18d640 1 -- 172.21.15.112:0/2637106504 wait complete. 2024-04-25T00:55:43.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.128+0000 7f4bdebba640 1 Processor -- start 2024-04-25T00:55:43.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdebba640 1 -- start start 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdebba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80add70 0x7f4bd8145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdebba640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bd80b3ce0 0x7f4bd814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdebba640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd80aeec0 con 0x7f4bd80b23f0 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdebba640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd80af780 con 0x7f4bd80b3ce0 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdebba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd80b4170 con 0x7f4bd80add70 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdd157640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bd80b3ce0 0x7f4bd814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdd157640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bd80b3ce0 0x7f4bd814fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50980/0 (socket says 172.21.15.112:50980) 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdd157640 1 -- 172.21.15.112:0/117883864 learned_addr learned my addr 172.21.15.112:0/117883864 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bcffff640 1 --2- 172.21.15.112:0/117883864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80add70 0x7f4bd8145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdd157640 1 -- 172.21.15.112:0/117883864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80add70 msgr2=0x7f4bd8145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:43.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdd157640 1 --2- 172.21.15.112:0/117883864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80add70 0x7f4bd8145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdd157640 1 -- 172.21.15.112:0/117883864 >> v1:172.21.15.187:6789/0 conn(0x7f4bd80b23f0 legacy=0x7f4bd8072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:43.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.132+0000 7f4bdd157640 1 -- 172.21.15.112:0/117883864 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd80b27d0 con 0x7f4bd80b3ce0 2024-04-25T00:55:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdd157640 1 --2- 172.21.15.112:0/117883864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80b3ce0 0x7f4bd814fb40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009e50 tx=0x7f4bd001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37279baf361e8b1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bcf7fe640 1 -- 172.21.15.112:0/117883864 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00a5780 con 0x7f4bd80b3ce0 2024-04-25T00:55:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bcf7fe640 1 -- 172.21.15.112:0/117883864 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd00a5920 con 0x7f4bd80b3ce0 2024-04-25T00:55:43.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bcf7fe640 1 -- 172.21.15.112:0/117883864 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00a5cf0 con 0x7f4bd80b3ce0 2024-04-25T00:55:43.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- 172.21.15.112:0/117883864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80b3ce0 msgr2=0x7f4bd814fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 --2- 172.21.15.112:0/117883864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80b3ce0 0x7f4bd814fb40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009e50 tx=0x7f4bd001c9f0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- 172.21.15.112:0/117883864 shutdown_connections 2024-04-25T00:55:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 --2- 172.21.15.112:0/117883864 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80b3ce0 0x7f4bd814fb40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 --2- 172.21.15.112:0/117883864 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80add70 0x7f4bd8145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- 172.21.15.112:0/117883864 >> 172.21.15.112:0/117883864 conn(0x7f4bd80b5980 msgr2=0x7f4bd80b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- 172.21.15.112:0/117883864 shutdown_connections 2024-04-25T00:55:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- 172.21.15.112:0/117883864 wait complete. 2024-04-25T00:55:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 Processor -- start 2024-04-25T00:55:43.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- start start 2024-04-25T00:55:43.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80add70 0x7f4bd813ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd80b23f0 0x7f4bd8139250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80b3ce0 0x7f4bd8135850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd80b52d0 con 0x7f4bd80add70 2024-04-25T00:55:43.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd80b5150 con 0x7f4bd80b23f0 2024-04-25T00:55:43.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd81528a0 con 0x7f4bd80b3ce0 2024-04-25T00:55:43.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80add70 0x7f4bd813ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80add70 0x7f4bd813ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:50984/0 (socket says 172.21.15.112:50984) 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 -- 172.21.15.112:0/609262931 learned_addr learned my addr 172.21.15.112:0/609262931 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bcffff640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd80b23f0 0x7f4bd8139250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdd157640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80b3ce0 0x7f4bd8135850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 -- 172.21.15.112:0/609262931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80b3ce0 msgr2=0x7f4bd8135850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80b3ce0 0x7f4bd8135850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 -- 172.21.15.112:0/609262931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd80b23f0 msgr2=0x7f4bd8139250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd80b23f0 0x7f4bd8139250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 -- 172.21.15.112:0/609262931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd813efd0 con 0x7f4bd80add70 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdc956640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80add70 0x7f4bd813ea90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400c450 tx=0x7f4bd400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bcd7fa640 1 -- 172.21.15.112:0/609262931 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4012830 con 0x7f4bd80add70 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bcd7fa640 1 -- 172.21.15.112:0/609262931 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd4017070 con 0x7f4bd80add70 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bcd7fa640 1 -- 172.21.15.112:0/609262931 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4012c30 con 0x7f4bd80add70 2024-04-25T00:55:43.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd813f280 con 0x7f4bd80add70 2024-04-25T00:55:43.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.136+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd814f020 con 0x7f4bd80add70 2024-04-25T00:55:43.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.140+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd804f580 con 0x7f4bd80add70 2024-04-25T00:55:43.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.140+0000 7f4bcd7fa640 1 -- 172.21.15.112:0/609262931 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4bd401c460 con 0x7f4bd80add70 2024-04-25T00:55:43.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.140+0000 7f4bcd7fa640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc40678c0 0x7f4bc4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.140+0000 7f4bcd7fa640 1 -- 172.21.15.112:0/609262931 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4bd4085970 con 0x7f4bd80add70 2024-04-25T00:55:43.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.140+0000 7f4bcffff640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc40678c0 0x7f4bc4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.140+0000 7f4bcffff640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc40678c0 0x7f4bc4069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80b2140 tx=0x7f4bc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.144+0000 7f4bcd7fa640 1 -- 172.21.15.112:0/609262931 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4bd4056540 con 0x7f4bd80add70 2024-04-25T00:55:43.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.304+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bd81e3570 con 0x7f4bd80add70 2024-04-25T00:55:43.309 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.304+0000 7f4bcd7fa640 1 -- 172.21.15.112:0/609262931 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f4bd4055e40 con 0x7f4bd80add70 2024-04-25T00:55:43.310 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:43.310 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:43.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc40678c0 msgr2=0x7f4bc4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc40678c0 0x7f4bc4069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80b2140 tx=0x7f4bc8008040 comp rx=0 tx=0).stop 2024-04-25T00:55:43.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80add70 msgr2=0x7f4bd813ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.312 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80add70 0x7f4bd813ea90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400c450 tx=0x7f4bd400c910 comp rx=0 tx=0).stop 2024-04-25T00:55:43.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 shutdown_connections 2024-04-25T00:55:43.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd80b3ce0 0x7f4bd8135850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc40678c0 0x7f4bc4069d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd80b23f0 0x7f4bd8139250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 --2- 172.21.15.112:0/609262931 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd80add70 0x7f4bd813ea90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 >> 172.21.15.112:0/609262931 conn(0x7f4bd80b5980 msgr2=0x7f4bd806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:43.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 shutdown_connections 2024-04-25T00:55:43.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.308+0000 7f4bdebba640 1 -- 172.21.15.112:0/609262931 wait complete. 2024-04-25T00:55:43.329 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:55:43.329 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:55:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.444+0000 7f0bd3491640 1 Processor -- start 2024-04-25T00:55:43.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.444+0000 7f0bd3491640 1 -- start start 2024-04-25T00:55:43.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd3491640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0bcc0b5790 0x7f0bcc150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd3491640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bcc067d70 con 0x7f0bcc072530 2024-04-25T00:55:43.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd3491640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bcc0b4350 con 0x7f0bcc0b3a80 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd3491640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0bcc072910 con 0x7f0bcc0b5790 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd1a2e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0bcc0b5790 0x7f0bcc150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd122d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0bcc072530 0x7f0bcc066f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57206/0 (socket says 172.21.15.112:57206) 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd122d640 1 -- 172.21.15.112:0/591426806 learned_addr learned my addr 172.21.15.112:0/591426806 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bcbfff640 1 -- 172.21.15.112:0/591426806 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109293173 0 0) 0x7f0bcc067d70 con 0x7f0bcc072530 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bcbfff640 1 -- 172.21.15.112:0/591426806 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bb4003530 con 0x7f0bcc072530 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd1a2e640 1 -- 172.21.15.112:0/591426806 >> v1:172.21.15.187:6789/0 conn(0x7f0bcc0b3a80 legacy=0x7f0bcc067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd1a2e640 1 -- 172.21.15.112:0/591426806 >> v1:172.21.15.112:6789/0 conn(0x7f0bcc072530 legacy=0x7f0bcc066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd1a2e640 1 -- 172.21.15.112:0/591426806 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc0b4c40 con 0x7f0bcc0b5790 2024-04-25T00:55:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bd1a2e640 1 --2- 172.21.15.112:0/591426806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc0b5790 0x7f0bcc150ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0097c0 tx=0x7f0bbc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155477a963a6ab07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bcbfff640 1 -- 172.21.15.112:0/591426806 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0b0070 con 0x7f0bcc0b5790 2024-04-25T00:55:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bcbfff640 1 -- 172.21.15.112:0/591426806 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc0ab600 con 0x7f0bcc0b5790 2024-04-25T00:55:43.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.448+0000 7f0bcbfff640 1 -- 172.21.15.112:0/591426806 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0ab9d0 con 0x7f0bcc0b5790 2024-04-25T00:55:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- 172.21.15.112:0/591426806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc0b5790 msgr2=0x7f0bcc150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 --2- 172.21.15.112:0/591426806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc0b5790 0x7f0bcc150ca0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0097c0 tx=0x7f0bbc01c380 comp rx=0 tx=0).stop 2024-04-25T00:55:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- 172.21.15.112:0/591426806 shutdown_connections 2024-04-25T00:55:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 --2- 172.21.15.112:0/591426806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc0b5790 0x7f0bcc150ca0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- 172.21.15.112:0/591426806 >> 172.21.15.112:0/591426806 conn(0x7f0bcc06b210 msgr2=0x7f0bcc06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- 172.21.15.112:0/591426806 shutdown_connections 2024-04-25T00:55:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- 172.21.15.112:0/591426806 wait complete. 2024-04-25T00:55:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 Processor -- start 2024-04-25T00:55:43.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- start start 2024-04-25T00:55:43.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc072530 0x7f0bcc13f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bcc0b3a80 0x7f0bcc1317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bcc0b5790 0x7f0bcc1e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b7250 con 0x7f0bcc072530 2024-04-25T00:55:43.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b3e90 con 0x7f0bcc0b3a80 2024-04-25T00:55:43.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc0b73d0 con 0x7f0bcc0b5790 2024-04-25T00:55:43.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc072530 0x7f0bcc13f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.457 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc072530 0x7f0bcc13f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51012/0 (socket says 172.21.15.112:51012) 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 -- 172.21.15.112:0/1602787715 learned_addr learned my addr 172.21.15.112:0/1602787715 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 -- 172.21.15.112:0/1602787715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bcc0b5790 msgr2=0x7f0bcc1e0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd1a2e640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bcc0b5790 0x7f0bcc1e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bcc0b5790 0x7f0bcc1e0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 -- 172.21.15.112:0/1602787715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bcc0b3a80 msgr2=0x7f0bcc1317a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd0a2c640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bcc0b3a80 0x7f0bcc1317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bcc0b3a80 0x7f0bcc1317a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 -- 172.21.15.112:0/1602787715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc140160 con 0x7f0bcc072530 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd122d640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc072530 0x7f0bcc13f2c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400cbf0 tx=0x7f0bc400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bc9ffb640 1 -- 172.21.15.112:0/1602787715 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4017070 con 0x7f0bcc072530 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bc9ffb640 1 -- 172.21.15.112:0/1602787715 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc400ace0 con 0x7f0bcc072530 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc137c60 con 0x7f0bcc072530 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bc9ffb640 1 -- 172.21.15.112:0/1602787715 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4012620 con 0x7f0bcc072530 2024-04-25T00:55:43.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc1e1c80 con 0x7f0bcc072530 2024-04-25T00:55:43.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.452+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc04f560 con 0x7f0bcc072530 2024-04-25T00:55:43.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.456+0000 7f0bc9ffb640 1 -- 172.21.15.112:0/1602787715 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0bc401a070 con 0x7f0bcc072530 2024-04-25T00:55:43.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.456+0000 7f0bc9ffb640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0bb80678c0 0x7f0bb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.456+0000 7f0bc9ffb640 1 -- 172.21.15.112:0/1602787715 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0bc4085a20 con 0x7f0bcc072530 2024-04-25T00:55:43.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.456+0000 7f0bd0a2c640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0bb80678c0 0x7f0bb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.460 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.456+0000 7f0bd0a2c640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0bb80678c0 0x7f0bb8069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0007fd0 tx=0x7f0bc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.463 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.460+0000 7f0bc9ffb640 1 -- 172.21.15.112:0/1602787715 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0bc40565f0 con 0x7f0bcc072530 2024-04-25T00:55:43.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.616+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bcc13fea0 con 0x7f0bb80678c0 2024-04-25T00:55:43.638 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.632+0000 7f0bc9ffb640 1 -- 172.21.15.112:0/1602787715 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f0bc000d450 con 0x7f0bb80678c0 2024-04-25T00:55:43.640 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:43.642 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:55:43.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0bb80678c0 msgr2=0x7f0bb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0bb80678c0 0x7f0bb8069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0007fd0 tx=0x7f0bc0009040 comp rx=0 tx=0).stop 2024-04-25T00:55:43.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc072530 msgr2=0x7f0bcc13f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc072530 0x7f0bcc13f2c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400cbf0 tx=0x7f0bc400a430 comp rx=0 tx=0).stop 2024-04-25T00:55:43.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 shutdown_connections 2024-04-25T00:55:43.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bcc0b5790 0x7f0bcc1e0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0bb80678c0 0x7f0bb8069d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bcc0b3a80 0x7f0bcc1317a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 --2- 172.21.15.112:0/1602787715 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bcc072530 0x7f0bcc13f2c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 >> 172.21.15.112:0/1602787715 conn(0x7f0bcc06b210 msgr2=0x7f0bcc0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:43.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 shutdown_connections 2024-04-25T00:55:43.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.640+0000 7f0bd3491640 1 -- 172.21.15.112:0/1602787715 wait complete. 2024-04-25T00:55:43.661 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T00:55:43.514219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.935199"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018125+0000","last_change":"2024-04-25T00:55:39.661241+0000","last_active":"2024-04-25T00:55:41.018125+0000","last_peered":"2024-04-25T00:55:41.018125+0000","last_clean":"2024-04-25T00:55:41.018125+0000","last_became_active":"2024-04-25T00:55:39.660150+0000","last_became_peered":"2024-04-25T00:55:39.660150+0000","last_unstale":"2024-04-25T00:55:41.018125+0000","last_undegraded":"2024-04-25T00:55:41.018125+0000","last_fullsized":"2024-04-25T00:55:41.018125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:11.034367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637050+0000","last_change":"2024-04-25T00:55:39.633699+0000","last_active":"2024-04-25T00:55:40.637050+0000","last_peered":"2024-04-25T00:55:40.637050+0000","last_clean":"2024-04-25T00:55:40.637050+0000","last_became_active":"2024-04-25T00:55:39.632942+0000","last_became_peered":"2024-04-25T00:55:39.632942+0000","last_unstale":"2024-04-25T00:55:40.637050+0000","last_undegraded":"2024-04-25T00:55:40.637050+0000","last_fullsized":"2024-04-25T00:55:40.637050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018670+0000","last_change":"2024-04-25T00:55:39.629762+0000","last_active":"2024-04-25T00:55:41.018670+0000","last_peered":"2024-04-25T00:55:41.018670+0000","last_clean":"2024-04-25T00:55:41.018670+0000","last_became_active":"2024-04-25T00:55:39.628798+0000","last_became_peered":"2024-04-25T00:55:39.628798+0000","last_unstale":"2024-04-25T00:55:41.018670+0000","last_undegraded":"2024-04-25T00:55:41.018670+0000","last_fullsized":"2024-04-25T00:55:41.018670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:23.939072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637845+0000","last_change":"2024-04-25T00:55:39.643398+0000","last_active":"2024-04-25T00:55:40.637845+0000","last_peered":"2024-04-25T00:55:40.637845+0000","last_clean":"2024-04-25T00:55:40.637845+0000","last_became_active":"2024-04-25T00:55:39.642844+0000","last_became_peered":"2024-04-25T00:55:39.642844+0000","last_unstale":"2024-04-25T00:55:40.637845+0000","last_undegraded":"2024-04-25T00:55:40.637845+0000","last_fullsized":"2024-04-25T00:55:40.637845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017483+0000","last_change":"2024-04-25T00:55:39.649899+0000","last_active":"2024-04-25T00:55:41.017483+0000","last_peered":"2024-04-25T00:55:41.017483+0000","last_clean":"2024-04-25T00:55:41.017483+0000","last_became_active":"2024-04-25T00:55:39.648852+0000","last_became_peered":"2024-04-25T00:55:39.648852+0000","last_unstale":"2024-04-25T00:55:41.017483+0000","last_undegraded":"2024-04-25T00:55:41.017483+0000","last_fullsized":"2024-04-25T00:55:41.017483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638862+0000","last_change":"2024-04-25T00:55:39.635369+0000","last_active":"2024-04-25T00:55:40.638862+0000","last_peered":"2024-04-25T00:55:40.638862+0000","last_clean":"2024-04-25T00:55:40.638862+0000","last_became_active":"2024-04-25T00:55:39.634678+0000","last_became_peered":"2024-04-25T00:55:39.634678+0000","last_unstale":"2024-04-25T00:55:40.638862+0000","last_undegraded":"2024-04-25T00:55:40.638862+0000","last_fullsized":"2024-04-25T00:55:40.638862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:30.556421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016962+0000","last_change":"2024-04-25T00:55:39.646309+0000","last_active":"2024-04-25T00:55:41.016962+0000","last_peered":"2024-04-25T00:55:41.016962+0000","last_clean":"2024-04-25T00:55:41.016962+0000","last_became_active":"2024-04-25T00:55:39.645778+0000","last_became_peered":"2024-04-25T00:55:39.645778+0000","last_unstale":"2024-04-25T00:55:41.016962+0000","last_undegraded":"2024-04-25T00:55:41.016962+0000","last_fullsized":"2024-04-25T00:55:41.016962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020590+0000","last_change":"2024-04-25T00:55:39.643250+0000","last_active":"2024-04-25T00:55:41.020590+0000","last_peered":"2024-04-25T00:55:41.020590+0000","last_clean":"2024-04-25T00:55:41.020590+0000","last_became_active":"2024-04-25T00:55:39.642948+0000","last_became_peered":"2024-04-25T00:55:39.642948+0000","last_unstale":"2024-04-25T00:55:41.020590+0000","last_undegraded":"2024-04-25T00:55:41.020590+0000","last_fullsized":"2024-04-25T00:55:41.020590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:53.762656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018212+0000","last_change":"2024-04-25T00:55:39.658665+0000","last_active":"2024-04-25T00:55:41.018212+0000","last_peered":"2024-04-25T00:55:41.018212+0000","last_clean":"2024-04-25T00:55:41.018212+0000","last_became_active":"2024-04-25T00:55:39.658276+0000","last_became_peered":"2024-04-25T00:55:39.658276+0000","last_unstale":"2024-04-25T00:55:41.018212+0000","last_undegraded":"2024-04-25T00:55:41.018212+0000","last_fullsized":"2024-04-25T00:55:41.018212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:47.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652697+0000","last_change":"2024-04-25T00:55:39.660203+0000","last_active":"2024-04-25T00:55:40.652697+0000","last_peered":"2024-04-25T00:55:40.652697+0000","last_clean":"2024-04-25T00:55:40.652697+0000","last_became_active":"2024-04-25T00:55:39.656617+0000","last_became_peered":"2024-04-25T00:55:39.656617+0000","last_unstale":"2024-04-25T00:55:40.652697+0000","last_undegraded":"2024-04-25T00:55:40.652697+0000","last_fullsized":"2024-04-25T00:55:40.652697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652875+0000","last_change":"2024-04-25T00:55:39.638495+0000","last_active":"2024-04-25T00:55:40.652875+0000","last_peered":"2024-04-25T00:55:40.652875+0000","last_clean":"2024-04-25T00:55:40.652875+0000","last_became_active":"2024-04-25T00:55:39.631516+0000","last_became_peered":"2024-04-25T00:55:39.631516+0000","last_unstale":"2024-04-25T00:55:40.652875+0000","last_undegraded":"2024-04-25T00:55:40.652875+0000","last_fullsized":"2024-04-25T00:55:40.652875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636317+0000","last_change":"2024-04-25T00:55:39.657591+0000","last_active":"2024-04-25T00:55:40.636317+0000","last_peered":"2024-04-25T00:55:40.636317+0000","last_clean":"2024-04-25T00:55:40.636317+0000","last_became_active":"2024-04-25T00:55:39.657259+0000","last_became_peered":"2024-04-25T00:55:39.657259+0000","last_unstale":"2024-04-25T00:55:40.636317+0000","last_undegraded":"2024-04-25T00:55:40.636317+0000","last_fullsized":"2024-04-25T00:55:40.636317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:56.786582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.022200+0000","last_change":"2024-04-25T00:55:39.656495+0000","last_active":"2024-04-25T00:55:41.022200+0000","last_peered":"2024-04-25T00:55:41.022200+0000","last_clean":"2024-04-25T00:55:41.022200+0000","last_became_active":"2024-04-25T00:55:39.655994+0000","last_became_peered":"2024-04-25T00:55:39.655994+0000","last_unstale":"2024-04-25T00:55:41.022200+0000","last_undegraded":"2024-04-25T00:55:41.022200+0000","last_fullsized":"2024-04-25T00:55:41.022200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019282+0000","last_change":"2024-04-25T00:55:39.636352+0000","last_active":"2024-04-25T00:55:41.019282+0000","last_peered":"2024-04-25T00:55:41.019282+0000","last_clean":"2024-04-25T00:55:41.019282+0000","last_became_active":"2024-04-25T00:55:39.636013+0000","last_became_peered":"2024-04-25T00:55:39.636013+0000","last_unstale":"2024-04-25T00:55:41.019282+0000","last_undegraded":"2024-04-25T00:55:41.019282+0000","last_fullsized":"2024-04-25T00:55:41.019282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636741+0000","last_change":"2024-04-25T00:55:39.634943+0000","last_active":"2024-04-25T00:55:40.636741+0000","last_peered":"2024-04-25T00:55:40.636741+0000","last_clean":"2024-04-25T00:55:40.636741+0000","last_became_active":"2024-04-25T00:55:39.634116+0000","last_became_peered":"2024-04-25T00:55:39.634116+0000","last_unstale":"2024-04-25T00:55:40.636741+0000","last_undegraded":"2024-04-25T00:55:40.636741+0000","last_fullsized":"2024-04-25T00:55:40.636741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637210+0000","last_change":"2024-04-25T00:55:39.630149+0000","last_active":"2024-04-25T00:55:40.637210+0000","last_peered":"2024-04-25T00:55:40.637210+0000","last_clean":"2024-04-25T00:55:40.637210+0000","last_became_active":"2024-04-25T00:55:39.629387+0000","last_became_peered":"2024-04-25T00:55:39.629387+0000","last_unstale":"2024-04-25T00:55:40.637210+0000","last_undegraded":"2024-04-25T00:55:40.637210+0000","last_fullsized":"2024-04-25T00:55:40.637210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638541+0000","last_change":"2024-04-25T00:55:39.650995+0000","last_active":"2024-04-25T00:55:40.638541+0000","last_peered":"2024-04-25T00:55:40.638541+0000","last_clean":"2024-04-25T00:55:40.638541+0000","last_became_active":"2024-04-25T00:55:39.650711+0000","last_became_peered":"2024-04-25T00:55:39.650711+0000","last_unstale":"2024-04-25T00:55:40.638541+0000","last_undegraded":"2024-04-25T00:55:40.638541+0000","last_fullsized":"2024-04-25T00:55:40.638541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.187193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019011+0000","last_change":"2024-04-25T00:55:39.656754+0000","last_active":"2024-04-25T00:55:41.019011+0000","last_peered":"2024-04-25T00:55:41.019011+0000","last_clean":"2024-04-25T00:55:41.019011+0000","last_became_active":"2024-04-25T00:55:39.656397+0000","last_became_peered":"2024-04-25T00:55:39.656397+0000","last_unstale":"2024-04-25T00:55:41.019011+0000","last_undegraded":"2024-04-25T00:55:41.019011+0000","last_fullsized":"2024-04-25T00:55:41.019011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.087648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018205+0000","last_change":"2024-04-25T00:55:39.648560+0000","last_active":"2024-04-25T00:55:41.018205+0000","last_peered":"2024-04-25T00:55:41.018205+0000","last_clean":"2024-04-25T00:55:41.018205+0000","last_became_active":"2024-04-25T00:55:39.647080+0000","last_became_peered":"2024-04-25T00:55:39.647080+0000","last_unstale":"2024-04-25T00:55:41.018205+0000","last_undegraded":"2024-04-25T00:55:41.018205+0000","last_fullsized":"2024-04-25T00:55:41.018205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:22.249474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.653020+0000","last_change":"2024-04-25T00:55:39.659058+0000","last_active":"2024-04-25T00:55:40.653020+0000","last_peered":"2024-04-25T00:55:40.653020+0000","last_clean":"2024-04-25T00:55:40.653020+0000","last_became_active":"2024-04-25T00:55:39.657152+0000","last_became_peered":"2024-04-25T00:55:39.657152+0000","last_unstale":"2024-04-25T00:55:40.653020+0000","last_undegraded":"2024-04-25T00:55:40.653020+0000","last_fullsized":"2024-04-25T00:55:40.653020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020095+0000","last_change":"2024-04-25T00:55:39.650890+0000","last_active":"2024-04-25T00:55:41.020095+0000","last_peered":"2024-04-25T00:55:41.020095+0000","last_clean":"2024-04-25T00:55:41.020095+0000","last_became_active":"2024-04-25T00:55:39.650235+0000","last_became_peered":"2024-04-25T00:55:39.650235+0000","last_unstale":"2024-04-25T00:55:41.020095+0000","last_undegraded":"2024-04-25T00:55:41.020095+0000","last_fullsized":"2024-04-25T00:55:41.020095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638004+0000","last_change":"2024-04-25T00:55:39.646725+0000","last_active":"2024-04-25T00:55:40.638004+0000","last_peered":"2024-04-25T00:55:40.638004+0000","last_clean":"2024-04-25T00:55:40.638004+0000","last_became_active":"2024-04-25T00:55:39.646192+0000","last_became_peered":"2024-04-25T00:55:39.646192+0000","last_unstale":"2024-04-25T00:55:40.638004+0000","last_undegraded":"2024-04-25T00:55:40.638004+0000","last_fullsized":"2024-04-25T00:55:40.638004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:51.008333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020594+0000","last_change":"2024-04-25T00:55:39.642303+0000","last_active":"2024-04-25T00:55:41.020594+0000","last_peered":"2024-04-25T00:55:41.020594+0000","last_clean":"2024-04-25T00:55:41.020594+0000","last_became_active":"2024-04-25T00:55:39.641627+0000","last_became_peered":"2024-04-25T00:55:39.641627+0000","last_unstale":"2024-04-25T00:55:41.020594+0000","last_undegraded":"2024-04-25T00:55:41.020594+0000","last_fullsized":"2024-04-25T00:55:41.020594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637271+0000","last_change":"2024-04-25T00:55:39.643919+0000","last_active":"2024-04-25T00:55:40.637271+0000","last_peered":"2024-04-25T00:55:40.637271+0000","last_clean":"2024-04-25T00:55:40.637271+0000","last_became_active":"2024-04-25T00:55:39.643196+0000","last_became_peered":"2024-04-25T00:55:39.643196+0000","last_unstale":"2024-04-25T00:55:40.637271+0000","last_undegraded":"2024-04-25T00:55:40.637271+0000","last_fullsized":"2024-04-25T00:55:40.637271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021581+0000","last_change":"2024-04-25T00:55:39.641462+0000","last_active":"2024-04-25T00:55:41.021581+0000","last_peered":"2024-04-25T00:55:41.021581+0000","last_clean":"2024-04-25T00:55:41.021581+0000","last_became_active":"2024-04-25T00:55:39.640648+0000","last_became_peered":"2024-04-25T00:55:39.640648+0000","last_unstale":"2024-04-25T00:55:41.021581+0000","last_undegraded":"2024-04-25T00:55:41.021581+0000","last_fullsized":"2024-04-25T00:55:41.021581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652767+0000","last_change":"2024-04-25T00:55:39.648816+0000","last_active":"2024-04-25T00:55:40.652767+0000","last_peered":"2024-04-25T00:55:40.652767+0000","last_clean":"2024-04-25T00:55:40.652767+0000","last_became_active":"2024-04-25T00:55:39.648337+0000","last_became_peered":"2024-04-25T00:55:39.648337+0000","last_unstale":"2024-04-25T00:55:40.652767+0000","last_undegraded":"2024-04-25T00:55:40.652767+0000","last_fullsized":"2024-04-25T00:55:40.652767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637430+0000","last_change":"2024-04-25T00:55:39.644333+0000","last_active":"2024-04-25T00:55:40.637430+0000","last_peered":"2024-04-25T00:55:40.637430+0000","last_clean":"2024-04-25T00:55:40.637430+0000","last_became_active":"2024-04-25T00:55:39.643706+0000","last_became_peered":"2024-04-25T00:55:39.643706+0000","last_unstale":"2024-04-25T00:55:40.637430+0000","last_undegraded":"2024-04-25T00:55:40.637430+0000","last_fullsized":"2024-04-25T00:55:40.637430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021853+0000","last_change":"2024-04-25T00:55:39.631560+0000","last_active":"2024-04-25T00:55:41.021853+0000","last_peered":"2024-04-25T00:55:41.021853+0000","last_clean":"2024-04-25T00:55:41.021853+0000","last_became_active":"2024-04-25T00:55:39.630796+0000","last_became_peered":"2024-04-25T00:55:39.630796+0000","last_unstale":"2024-04-25T00:55:41.021853+0000","last_undegraded":"2024-04-25T00:55:41.021853+0000","last_fullsized":"2024-04-25T00:55:41.021853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020172+0000","last_change":"2024-04-25T00:55:39.643191+0000","last_active":"2024-04-25T00:55:41.020172+0000","last_peered":"2024-04-25T00:55:41.020172+0000","last_clean":"2024-04-25T00:55:41.020172+0000","last_became_active":"2024-04-25T00:55:39.642846+0000","last_became_peered":"2024-04-25T00:55:39.642846+0000","last_unstale":"2024-04-25T00:55:41.020172+0000","last_undegraded":"2024-04-25T00:55:41.020172+0000","last_fullsized":"2024-04-25T00:55:41.020172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638416+0000","last_change":"2024-04-25T00:55:39.643174+0000","last_active":"2024-04-25T00:55:40.638416+0000","last_peered":"2024-04-25T00:55:40.638416+0000","last_clean":"2024-04-25T00:55:40.638416+0000","last_became_active":"2024-04-25T00:55:39.642575+0000","last_became_peered":"2024-04-25T00:55:39.642575+0000","last_unstale":"2024-04-25T00:55:40.638416+0000","last_undegraded":"2024-04-25T00:55:40.638416+0000","last_fullsized":"2024-04-25T00:55:40.638416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:47.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018168+0000","last_change":"2024-04-25T00:55:39.627384+0000","last_active":"2024-04-25T00:55:41.018168+0000","last_peered":"2024-04-25T00:55:41.018168+0000","last_clean":"2024-04-25T00:55:41.018168+0000","last_became_active":"2024-04-25T00:55:39.627078+0000","last_became_peered":"2024-04-25T00:55:39.627078+0000","last_unstale":"2024-04-25T00:55:41.018168+0000","last_undegraded":"2024-04-25T00:55:41.018168+0000","last_fullsized":"2024-04-25T00:55:41.018168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:21.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652573+0000","last_change":"2024-04-25T00:55:39.655121+0000","last_active":"2024-04-25T00:55:40.652573+0000","last_peered":"2024-04-25T00:55:40.652573+0000","last_clean":"2024-04-25T00:55:40.652573+0000","last_became_active":"2024-04-25T00:55:39.654774+0000","last_became_peered":"2024-04-25T00:55:39.654774+0000","last_unstale":"2024-04-25T00:55:40.652573+0000","last_undegraded":"2024-04-25T00:55:40.652573+0000","last_fullsized":"2024-04-25T00:55:40.652573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636316+0000","last_change":"2024-04-25T00:55:39.634737+0000","last_active":"2024-04-25T00:55:40.636316+0000","last_peered":"2024-04-25T00:55:40.636316+0000","last_clean":"2024-04-25T00:55:40.636316+0000","last_became_active":"2024-04-25T00:55:39.633943+0000","last_became_peered":"2024-04-25T00:55:39.633943+0000","last_unstale":"2024-04-25T00:55:40.636316+0000","last_undegraded":"2024-04-25T00:55:40.636316+0000","last_fullsized":"2024-04-25T00:55:40.636316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:09.262927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021945+0000","last_change":"2024-04-25T00:55:38.629602+0000","last_active":"2024-04-25T00:55:41.021945+0000","last_peered":"2024-04-25T00:55:41.021945+0000","last_clean":"2024-04-25T00:55:41.021945+0000","last_became_active":"2024-04-25T00:55:38.628956+0000","last_became_peered":"2024-04-25T00:55:38.628956+0000","last_unstale":"2024-04-25T00:55:41.021945+0000","last_undegraded":"2024-04-25T00:55:41.021945+0000","last_fullsized":"2024-04-25T00:55:41.021945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:50.369672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020489+0000","last_change":"2024-04-25T00:55:39.630640+0000","last_active":"2024-04-25T00:55:41.020489+0000","last_peered":"2024-04-25T00:55:41.020489+0000","last_clean":"2024-04-25T00:55:41.020489+0000","last_became_active":"2024-04-25T00:55:39.629906+0000","last_became_peered":"2024-04-25T00:55:39.629906+0000","last_unstale":"2024-04-25T00:55:41.020489+0000","last_undegraded":"2024-04-25T00:55:41.020489+0000","last_fullsized":"2024-04-25T00:55:41.020489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652654+0000","last_change":"2024-04-25T00:55:38.626215+0000","last_active":"2024-04-25T00:55:40.652654+0000","last_peered":"2024-04-25T00:55:40.652654+0000","last_clean":"2024-04-25T00:55:40.652654+0000","last_became_active":"2024-04-25T00:55:38.625439+0000","last_became_peered":"2024-04-25T00:55:38.625439+0000","last_unstale":"2024-04-25T00:55:40.652654+0000","last_undegraded":"2024-04-25T00:55:40.652654+0000","last_fullsized":"2024-04-25T00:55:40.652654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020655+0000","last_change":"2024-04-25T00:55:38.626401+0000","last_active":"2024-04-25T00:55:41.020655+0000","last_peered":"2024-04-25T00:55:41.020655+0000","last_clean":"2024-04-25T00:55:41.020655+0000","last_became_active":"2024-04-25T00:55:38.625884+0000","last_became_peered":"2024-04-25T00:55:38.625884+0000","last_unstale":"2024-04-25T00:55:41.020655+0000","last_undegraded":"2024-04-25T00:55:41.020655+0000","last_fullsized":"2024-04-25T00:55:41.020655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020981+0000","last_change":"2024-04-25T00:55:39.638181+0000","last_active":"2024-04-25T00:55:41.020981+0000","last_peered":"2024-04-25T00:55:41.020981+0000","last_clean":"2024-04-25T00:55:41.020981+0000","last_became_active":"2024-04-25T00:55:39.637900+0000","last_became_peered":"2024-04-25T00:55:39.637900+0000","last_unstale":"2024-04-25T00:55:41.020981+0000","last_undegraded":"2024-04-25T00:55:41.020981+0000","last_fullsized":"2024-04-25T00:55:41.020981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637316+0000","last_change":"2024-04-25T00:55:38.621335+0000","last_active":"2024-04-25T00:55:40.637316+0000","last_peered":"2024-04-25T00:55:40.637316+0000","last_clean":"2024-04-25T00:55:40.637316+0000","last_became_active":"2024-04-25T00:55:38.620467+0000","last_became_peered":"2024-04-25T00:55:38.620467+0000","last_unstale":"2024-04-25T00:55:40.637316+0000","last_undegraded":"2024-04-25T00:55:40.637316+0000","last_fullsized":"2024-04-25T00:55:40.637316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636806+0000","last_change":"2024-04-25T00:55:39.642780+0000","last_active":"2024-04-25T00:55:40.636806+0000","last_peered":"2024-04-25T00:55:40.636806+0000","last_clean":"2024-04-25T00:55:40.636806+0000","last_became_active":"2024-04-25T00:55:39.642410+0000","last_became_peered":"2024-04-25T00:55:39.642410+0000","last_unstale":"2024-04-25T00:55:40.636806+0000","last_undegraded":"2024-04-25T00:55:40.636806+0000","last_fullsized":"2024-04-25T00:55:40.636806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652622+0000","last_change":"2024-04-25T00:55:38.634413+0000","last_active":"2024-04-25T00:55:40.652622+0000","last_peered":"2024-04-25T00:55:40.652622+0000","last_clean":"2024-04-25T00:55:40.652622+0000","last_became_active":"2024-04-25T00:55:38.633760+0000","last_became_peered":"2024-04-25T00:55:38.633760+0000","last_unstale":"2024-04-25T00:55:40.652622+0000","last_undegraded":"2024-04-25T00:55:40.652622+0000","last_fullsized":"2024-04-25T00:55:40.652622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637854+0000","last_change":"2024-04-25T00:55:39.644911+0000","last_active":"2024-04-25T00:55:40.637854+0000","last_peered":"2024-04-25T00:55:40.637854+0000","last_clean":"2024-04-25T00:55:40.637854+0000","last_became_active":"2024-04-25T00:55:39.644568+0000","last_became_peered":"2024-04-25T00:55:39.644568+0000","last_unstale":"2024-04-25T00:55:40.637854+0000","last_undegraded":"2024-04-25T00:55:40.637854+0000","last_fullsized":"2024-04-25T00:55:40.637854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:16.517320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637925+0000","last_change":"2024-04-25T00:55:38.637548+0000","last_active":"2024-04-25T00:55:40.637925+0000","last_peered":"2024-04-25T00:55:40.637925+0000","last_clean":"2024-04-25T00:55:40.637925+0000","last_became_active":"2024-04-25T00:55:38.636962+0000","last_became_peered":"2024-04-25T00:55:38.636962+0000","last_unstale":"2024-04-25T00:55:40.637925+0000","last_undegraded":"2024-04-25T00:55:40.637925+0000","last_fullsized":"2024-04-25T00:55:40.637925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020866+0000","last_change":"2024-04-25T00:55:39.628306+0000","last_active":"2024-04-25T00:55:41.020866+0000","last_peered":"2024-04-25T00:55:41.020866+0000","last_clean":"2024-04-25T00:55:41.020866+0000","last_became_active":"2024-04-25T00:55:39.627530+0000","last_became_peered":"2024-04-25T00:55:39.627530+0000","last_unstale":"2024-04-25T00:55:41.020866+0000","last_undegraded":"2024-04-25T00:55:41.020866+0000","last_fullsized":"2024-04-25T00:55:41.020866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637335+0000","last_change":"2024-04-25T00:55:38.640365+0000","last_active":"2024-04-25T00:55:40.637335+0000","last_peered":"2024-04-25T00:55:40.637335+0000","last_clean":"2024-04-25T00:55:40.637335+0000","last_became_active":"2024-04-25T00:55:38.639988+0000","last_became_peered":"2024-04-25T00:55:38.639988+0000","last_unstale":"2024-04-25T00:55:40.637335+0000","last_undegraded":"2024-04-25T00:55:40.637335+0000","last_fullsized":"2024-04-25T00:55:40.637335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636444+0000","last_change":"2024-04-25T00:55:39.647034+0000","last_active":"2024-04-25T00:55:40.636444+0000","last_peered":"2024-04-25T00:55:40.636444+0000","last_clean":"2024-04-25T00:55:40.636444+0000","last_became_active":"2024-04-25T00:55:39.644279+0000","last_became_peered":"2024-04-25T00:55:39.644279+0000","last_unstale":"2024-04-25T00:55:40.636444+0000","last_undegraded":"2024-04-25T00:55:40.636444+0000","last_fullsized":"2024-04-25T00:55:40.636444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:21.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652840+0000","last_change":"2024-04-25T00:55:38.626291+0000","last_active":"2024-04-25T00:55:40.652840+0000","last_peered":"2024-04-25T00:55:40.652840+0000","last_clean":"2024-04-25T00:55:40.652840+0000","last_became_active":"2024-04-25T00:55:38.625530+0000","last_became_peered":"2024-04-25T00:55:38.625530+0000","last_unstale":"2024-04-25T00:55:40.652840+0000","last_undegraded":"2024-04-25T00:55:40.652840+0000","last_fullsized":"2024-04-25T00:55:40.652840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017484+0000","last_change":"2024-04-25T00:55:39.649518+0000","last_active":"2024-04-25T00:55:41.017484+0000","last_peered":"2024-04-25T00:55:41.017484+0000","last_clean":"2024-04-25T00:55:41.017484+0000","last_became_active":"2024-04-25T00:55:39.648701+0000","last_became_peered":"2024-04-25T00:55:39.648701+0000","last_unstale":"2024-04-25T00:55:41.017484+0000","last_undegraded":"2024-04-25T00:55:41.017484+0000","last_fullsized":"2024-04-25T00:55:41.017484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.552805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020445+0000","last_change":"2024-04-25T00:55:38.610242+0000","last_active":"2024-04-25T00:55:41.020445+0000","last_peered":"2024-04-25T00:55:41.020445+0000","last_clean":"2024-04-25T00:55:41.020445+0000","last_became_active":"2024-04-25T00:55:38.609855+0000","last_became_peered":"2024-04-25T00:55:38.609855+0000","last_unstale":"2024-04-25T00:55:41.020445+0000","last_undegraded":"2024-04-25T00:55:41.020445+0000","last_fullsized":"2024-04-25T00:55:41.020445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019689+0000","last_change":"2024-04-25T00:55:39.646421+0000","last_active":"2024-04-25T00:55:41.019689+0000","last_peered":"2024-04-25T00:55:41.019689+0000","last_clean":"2024-04-25T00:55:41.019689+0000","last_became_active":"2024-04-25T00:55:39.646119+0000","last_became_peered":"2024-04-25T00:55:39.646119+0000","last_unstale":"2024-04-25T00:55:41.019689+0000","last_undegraded":"2024-04-25T00:55:41.019689+0000","last_fullsized":"2024-04-25T00:55:41.019689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638593+0000","last_change":"2024-04-25T00:55:38.632875+0000","last_active":"2024-04-25T00:55:40.638593+0000","last_peered":"2024-04-25T00:55:40.638593+0000","last_clean":"2024-04-25T00:55:40.638593+0000","last_became_active":"2024-04-25T00:55:38.631370+0000","last_became_peered":"2024-04-25T00:55:38.631370+0000","last_unstale":"2024-04-25T00:55:40.638593+0000","last_undegraded":"2024-04-25T00:55:40.638593+0000","last_fullsized":"2024-04-25T00:55:40.638593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017573+0000","last_change":"2024-04-25T00:55:39.661000+0000","last_active":"2024-04-25T00:55:41.017573+0000","last_peered":"2024-04-25T00:55:41.017573+0000","last_clean":"2024-04-25T00:55:41.017573+0000","last_became_active":"2024-04-25T00:55:39.660704+0000","last_became_peered":"2024-04-25T00:55:39.660704+0000","last_unstale":"2024-04-25T00:55:41.017573+0000","last_undegraded":"2024-04-25T00:55:41.017573+0000","last_fullsized":"2024-04-25T00:55:41.017573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:16.517320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652441+0000","last_change":"2024-04-25T00:55:38.614039+0000","last_active":"2024-04-25T00:55:40.652441+0000","last_peered":"2024-04-25T00:55:40.652441+0000","last_clean":"2024-04-25T00:55:40.652441+0000","last_became_active":"2024-04-25T00:55:38.612513+0000","last_became_peered":"2024-04-25T00:55:38.612513+0000","last_unstale":"2024-04-25T00:55:40.652441+0000","last_undegraded":"2024-04-25T00:55:40.652441+0000","last_fullsized":"2024-04-25T00:55:40.652441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652439+0000","last_change":"2024-04-25T00:55:39.639974+0000","last_active":"2024-04-25T00:55:40.652439+0000","last_peered":"2024-04-25T00:55:40.652439+0000","last_clean":"2024-04-25T00:55:40.652439+0000","last_became_active":"2024-04-25T00:55:39.639397+0000","last_became_peered":"2024-04-25T00:55:39.639397+0000","last_unstale":"2024-04-25T00:55:40.652439+0000","last_undegraded":"2024-04-25T00:55:40.652439+0000","last_fullsized":"2024-04-25T00:55:40.652439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017966+0000","last_change":"2024-04-25T00:55:38.601709+0000","last_active":"2024-04-25T00:55:41.017966+0000","last_peered":"2024-04-25T00:55:41.017966+0000","last_clean":"2024-04-25T00:55:41.017966+0000","last_became_active":"2024-04-25T00:55:38.601079+0000","last_became_peered":"2024-04-25T00:55:38.601079+0000","last_unstale":"2024-04-25T00:55:41.017966+0000","last_undegraded":"2024-04-25T00:55:41.017966+0000","last_fullsized":"2024-04-25T00:55:41.017966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:50.369672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016947+0000","last_change":"2024-04-25T00:55:39.652284+0000","last_active":"2024-04-25T00:55:41.016947+0000","last_peered":"2024-04-25T00:55:41.016947+0000","last_clean":"2024-04-25T00:55:41.016947+0000","last_became_active":"2024-04-25T00:55:39.651442+0000","last_became_peered":"2024-04-25T00:55:39.651442+0000","last_unstale":"2024-04-25T00:55:41.016947+0000","last_undegraded":"2024-04-25T00:55:41.016947+0000","last_fullsized":"2024-04-25T00:55:41.016947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636948+0000","last_change":"2024-04-25T00:55:38.603875+0000","last_active":"2024-04-25T00:55:40.636948+0000","last_peered":"2024-04-25T00:55:40.636948+0000","last_clean":"2024-04-25T00:55:40.636948+0000","last_became_active":"2024-04-25T00:55:38.602412+0000","last_became_peered":"2024-04-25T00:55:38.602412+0000","last_unstale":"2024-04-25T00:55:40.636948+0000","last_undegraded":"2024-04-25T00:55:40.636948+0000","last_fullsized":"2024-04-25T00:55:40.636948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:34.488773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018026+0000","last_change":"2024-04-25T00:55:39.651592+0000","last_active":"2024-04-25T00:55:41.018026+0000","last_peered":"2024-04-25T00:55:41.018026+0000","last_clean":"2024-04-25T00:55:41.018026+0000","last_became_active":"2024-04-25T00:55:39.651045+0000","last_became_peered":"2024-04-25T00:55:39.651045+0000","last_unstale":"2024-04-25T00:55:41.018026+0000","last_undegraded":"2024-04-25T00:55:41.018026+0000","last_fullsized":"2024-04-25T00:55:41.018026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:51.008333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018836+0000","last_change":"2024-04-25T00:55:38.613063+0000","last_active":"2024-04-25T00:55:41.018836+0000","last_peered":"2024-04-25T00:55:41.018836+0000","last_clean":"2024-04-25T00:55:41.018836+0000","last_became_active":"2024-04-25T00:55:38.612651+0000","last_became_peered":"2024-04-25T00:55:38.612651+0000","last_unstale":"2024-04-25T00:55:41.018836+0000","last_undegraded":"2024-04-25T00:55:41.018836+0000","last_fullsized":"2024-04-25T00:55:41.018836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:25.842602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018376+0000","last_change":"2024-04-25T00:55:39.661043+0000","last_active":"2024-04-25T00:55:41.018376+0000","last_peered":"2024-04-25T00:55:41.018376+0000","last_clean":"2024-04-25T00:55:41.018376+0000","last_became_active":"2024-04-25T00:55:39.660577+0000","last_became_peered":"2024-04-25T00:55:39.660577+0000","last_unstale":"2024-04-25T00:55:41.018376+0000","last_undegraded":"2024-04-25T00:55:41.018376+0000","last_fullsized":"2024-04-25T00:55:41.018376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.187193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637626+0000","last_change":"2024-04-25T00:55:38.610833+0000","last_active":"2024-04-25T00:55:40.637626+0000","last_peered":"2024-04-25T00:55:40.637626+0000","last_clean":"2024-04-25T00:55:40.637626+0000","last_became_active":"2024-04-25T00:55:38.610476+0000","last_became_peered":"2024-04-25T00:55:38.610476+0000","last_unstale":"2024-04-25T00:55:40.637626+0000","last_undegraded":"2024-04-25T00:55:40.637626+0000","last_fullsized":"2024-04-25T00:55:40.637626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638296+0000","last_change":"2024-04-25T00:55:39.643691+0000","last_active":"2024-04-25T00:55:40.638296+0000","last_peered":"2024-04-25T00:55:40.638296+0000","last_clean":"2024-04-25T00:55:40.638296+0000","last_became_active":"2024-04-25T00:55:39.643024+0000","last_became_peered":"2024-04-25T00:55:39.643024+0000","last_unstale":"2024-04-25T00:55:40.638296+0000","last_undegraded":"2024-04-25T00:55:40.638296+0000","last_fullsized":"2024-04-25T00:55:40.638296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:22.249474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020063+0000","last_change":"2024-04-25T00:55:38.623842+0000","last_active":"2024-04-25T00:55:41.020063+0000","last_peered":"2024-04-25T00:55:41.020063+0000","last_clean":"2024-04-25T00:55:41.020063+0000","last_became_active":"2024-04-25T00:55:38.623514+0000","last_became_peered":"2024-04-25T00:55:38.623514+0000","last_unstale":"2024-04-25T00:55:41.020063+0000","last_undegraded":"2024-04-25T00:55:41.020063+0000","last_fullsized":"2024-04-25T00:55:41.020063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017354+0000","last_change":"2024-04-25T00:55:39.658238+0000","last_active":"2024-04-25T00:55:41.017354+0000","last_peered":"2024-04-25T00:55:41.017354+0000","last_clean":"2024-04-25T00:55:41.017354+0000","last_became_active":"2024-04-25T00:55:39.657889+0000","last_became_peered":"2024-04-25T00:55:39.657889+0000","last_unstale":"2024-04-25T00:55:41.017354+0000","last_undegraded":"2024-04-25T00:55:41.017354+0000","last_fullsized":"2024-04-25T00:55:41.017354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016737+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:55:41.016737+0000","last_peered":"2024-04-25T00:55:41.016737+0000","last_clean":"2024-04-25T00:55:41.016737+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:55:41.016737+0000","last_undegraded":"2024-04-25T00:55:41.016737+0000","last_fullsized":"2024-04-25T00:55:41.016737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637451+0000","last_change":"2024-04-25T00:55:38.632700+0000","last_active":"2024-04-25T00:55:40.637451+0000","last_peered":"2024-04-25T00:55:40.637451+0000","last_clean":"2024-04-25T00:55:40.637451+0000","last_became_active":"2024-04-25T00:55:38.631901+0000","last_became_peered":"2024-04-25T00:55:38.631901+0000","last_unstale":"2024-04-25T00:55:40.637451+0000","last_undegraded":"2024-04-25T00:55:40.637451+0000","last_fullsized":"2024-04-25T00:55:40.637451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021271+0000","last_change":"2024-04-25T00:55:39.639013+0000","last_active":"2024-04-25T00:55:41.021271+0000","last_peered":"2024-04-25T00:55:41.021271+0000","last_clean":"2024-04-25T00:55:41.021271+0000","last_became_active":"2024-04-25T00:55:39.638189+0000","last_became_peered":"2024-04-25T00:55:39.638189+0000","last_unstale":"2024-04-25T00:55:41.021271+0000","last_undegraded":"2024-04-25T00:55:41.021271+0000","last_fullsized":"2024-04-25T00:55:41.021271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016724+0000","last_change":"2024-04-25T00:55:38.621598+0000","last_active":"2024-04-25T00:55:41.016724+0000","last_peered":"2024-04-25T00:55:41.016724+0000","last_clean":"2024-04-25T00:55:41.016724+0000","last_became_active":"2024-04-25T00:55:38.620844+0000","last_became_peered":"2024-04-25T00:55:38.620844+0000","last_unstale":"2024-04-25T00:55:41.016724+0000","last_undegraded":"2024-04-25T00:55:41.016724+0000","last_fullsized":"2024-04-25T00:55:41.016724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019793+0000","last_change":"2024-04-25T00:55:39.626650+0000","last_active":"2024-04-25T00:55:41.019793+0000","last_peered":"2024-04-25T00:55:41.019793+0000","last_clean":"2024-04-25T00:55:41.019793+0000","last_became_active":"2024-04-25T00:55:39.625876+0000","last_became_peered":"2024-04-25T00:55:39.625876+0000","last_unstale":"2024-04-25T00:55:41.019793+0000","last_undegraded":"2024-04-25T00:55:41.019793+0000","last_fullsized":"2024-04-25T00:55:41.019793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018447+0000","last_change":"2024-04-25T00:55:38.605319+0000","last_active":"2024-04-25T00:55:41.018447+0000","last_peered":"2024-04-25T00:55:41.018447+0000","last_clean":"2024-04-25T00:55:41.018447+0000","last_became_active":"2024-04-25T00:55:38.604920+0000","last_became_peered":"2024-04-25T00:55:38.604920+0000","last_unstale":"2024-04-25T00:55:41.018447+0000","last_undegraded":"2024-04-25T00:55:41.018447+0000","last_fullsized":"2024-04-25T00:55:41.018447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636431+0000","last_change":"2024-04-25T00:55:39.635799+0000","last_active":"2024-04-25T00:55:40.636431+0000","last_peered":"2024-04-25T00:55:40.636431+0000","last_clean":"2024-04-25T00:55:40.636431+0000","last_became_active":"2024-04-25T00:55:39.633616+0000","last_became_peered":"2024-04-25T00:55:39.633616+0000","last_unstale":"2024-04-25T00:55:40.636431+0000","last_undegraded":"2024-04-25T00:55:40.636431+0000","last_fullsized":"2024-04-25T00:55:40.636431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638712+0000","last_change":"2024-04-25T00:55:38.638178+0000","last_active":"2024-04-25T00:55:40.638712+0000","last_peered":"2024-04-25T00:55:40.638712+0000","last_clean":"2024-04-25T00:55:40.638712+0000","last_became_active":"2024-04-25T00:55:38.637419+0000","last_became_peered":"2024-04-25T00:55:38.637419+0000","last_unstale":"2024-04-25T00:55:40.638712+0000","last_undegraded":"2024-04-25T00:55:40.638712+0000","last_fullsized":"2024-04-25T00:55:40.638712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636532+0000","last_change":"2024-04-25T00:55:39.635022+0000","last_active":"2024-04-25T00:55:40.636532+0000","last_peered":"2024-04-25T00:55:40.636532+0000","last_clean":"2024-04-25T00:55:40.636532+0000","last_became_active":"2024-04-25T00:55:39.634107+0000","last_became_peered":"2024-04-25T00:55:39.634107+0000","last_unstale":"2024-04-25T00:55:40.636532+0000","last_undegraded":"2024-04-25T00:55:40.636532+0000","last_fullsized":"2024-04-25T00:55:40.636532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.087648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637039+0000","last_change":"2024-04-25T00:55:38.607260+0000","last_active":"2024-04-25T00:55:40.637039+0000","last_peered":"2024-04-25T00:55:40.637039+0000","last_clean":"2024-04-25T00:55:40.637039+0000","last_became_active":"2024-04-25T00:55:38.606910+0000","last_became_peered":"2024-04-25T00:55:38.606910+0000","last_unstale":"2024-04-25T00:55:40.637039+0000","last_undegraded":"2024-04-25T00:55:40.637039+0000","last_fullsized":"2024-04-25T00:55:40.637039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:25.842602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637823+0000","last_change":"2024-04-25T00:55:39.641228+0000","last_active":"2024-04-25T00:55:40.637823+0000","last_peered":"2024-04-25T00:55:40.637823+0000","last_clean":"2024-04-25T00:55:40.637823+0000","last_became_active":"2024-04-25T00:55:39.640434+0000","last_became_peered":"2024-04-25T00:55:39.640434+0000","last_unstale":"2024-04-25T00:55:40.637823+0000","last_undegraded":"2024-04-25T00:55:40.637823+0000","last_fullsized":"2024-04-25T00:55:40.637823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.552805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018046+0000","last_change":"2024-04-25T00:55:38.630316+0000","last_active":"2024-04-25T00:55:41.018046+0000","last_peered":"2024-04-25T00:55:41.018046+0000","last_clean":"2024-04-25T00:55:41.018046+0000","last_became_active":"2024-04-25T00:55:38.629417+0000","last_became_peered":"2024-04-25T00:55:38.629417+0000","last_unstale":"2024-04-25T00:55:41.018046+0000","last_undegraded":"2024-04-25T00:55:41.018046+0000","last_fullsized":"2024-04-25T00:55:41.018046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636661+0000","last_change":"2024-04-25T00:55:39.633030+0000","last_active":"2024-04-25T00:55:40.636661+0000","last_peered":"2024-04-25T00:55:40.636661+0000","last_clean":"2024-04-25T00:55:40.636661+0000","last_became_active":"2024-04-25T00:55:39.632300+0000","last_became_peered":"2024-04-25T00:55:39.632300+0000","last_unstale":"2024-04-25T00:55:40.636661+0000","last_undegraded":"2024-04-25T00:55:40.636661+0000","last_fullsized":"2024-04-25T00:55:40.636661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652457+0000","last_change":"2024-04-25T00:55:38.634050+0000","last_active":"2024-04-25T00:55:40.652457+0000","last_peered":"2024-04-25T00:55:40.652457+0000","last_clean":"2024-04-25T00:55:40.652457+0000","last_became_active":"2024-04-25T00:55:38.633672+0000","last_became_peered":"2024-04-25T00:55:38.633672+0000","last_unstale":"2024-04-25T00:55:40.652457+0000","last_undegraded":"2024-04-25T00:55:40.652457+0000","last_fullsized":"2024-04-25T00:55:40.652457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.022010+0000","last_change":"2024-04-25T00:55:39.652710+0000","last_active":"2024-04-25T00:55:41.022010+0000","last_peered":"2024-04-25T00:55:41.022010+0000","last_clean":"2024-04-25T00:55:41.022010+0000","last_became_active":"2024-04-25T00:55:39.652302+0000","last_became_peered":"2024-04-25T00:55:39.652302+0000","last_unstale":"2024-04-25T00:55:41.022010+0000","last_undegraded":"2024-04-25T00:55:41.022010+0000","last_fullsized":"2024-04-25T00:55:41.022010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652764+0000","last_change":"2024-04-25T00:55:38.636274+0000","last_active":"2024-04-25T00:55:40.652764+0000","last_peered":"2024-04-25T00:55:40.652764+0000","last_clean":"2024-04-25T00:55:40.652764+0000","last_became_active":"2024-04-25T00:55:38.635626+0000","last_became_peered":"2024-04-25T00:55:38.635626+0000","last_unstale":"2024-04-25T00:55:40.652764+0000","last_undegraded":"2024-04-25T00:55:40.652764+0000","last_fullsized":"2024-04-25T00:55:40.652764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018903+0000","last_change":"2024-04-25T00:55:39.635041+0000","last_active":"2024-04-25T00:55:41.018903+0000","last_peered":"2024-04-25T00:55:41.018903+0000","last_clean":"2024-04-25T00:55:41.018903+0000","last_became_active":"2024-04-25T00:55:39.634758+0000","last_became_peered":"2024-04-25T00:55:39.634758+0000","last_unstale":"2024-04-25T00:55:41.018903+0000","last_undegraded":"2024-04-25T00:55:41.018903+0000","last_fullsized":"2024-04-25T00:55:41.018903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:53.762656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016600+0000","last_change":"2024-04-25T00:55:38.630288+0000","last_active":"2024-04-25T00:55:41.016600+0000","last_peered":"2024-04-25T00:55:41.016600+0000","last_clean":"2024-04-25T00:55:41.016600+0000","last_became_active":"2024-04-25T00:55:38.629355+0000","last_became_peered":"2024-04-25T00:55:38.629355+0000","last_unstale":"2024-04-25T00:55:41.016600+0000","last_undegraded":"2024-04-25T00:55:41.016600+0000","last_fullsized":"2024-04-25T00:55:41.016600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636226+0000","last_change":"2024-04-25T00:55:39.623782+0000","last_active":"2024-04-25T00:55:40.636226+0000","last_peered":"2024-04-25T00:55:40.636226+0000","last_clean":"2024-04-25T00:55:40.636226+0000","last_became_active":"2024-04-25T00:55:39.622815+0000","last_became_peered":"2024-04-25T00:55:39.622815+0000","last_unstale":"2024-04-25T00:55:40.636226+0000","last_undegraded":"2024-04-25T00:55:40.636226+0000","last_fullsized":"2024-04-25T00:55:40.636226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:03.896351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637109+0000","last_change":"2024-04-25T00:55:38.618497+0000","last_active":"2024-04-25T00:55:40.637109+0000","last_peered":"2024-04-25T00:55:40.637109+0000","last_clean":"2024-04-25T00:55:40.637109+0000","last_became_active":"2024-04-25T00:55:38.618150+0000","last_became_peered":"2024-04-25T00:55:38.618150+0000","last_unstale":"2024-04-25T00:55:40.637109+0000","last_undegraded":"2024-04-25T00:55:40.637109+0000","last_fullsized":"2024-04-25T00:55:40.637109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021086+0000","last_change":"2024-04-25T00:55:39.640734+0000","last_active":"2024-04-25T00:55:41.021086+0000","last_peered":"2024-04-25T00:55:41.021086+0000","last_clean":"2024-04-25T00:55:41.021086+0000","last_became_active":"2024-04-25T00:55:39.639372+0000","last_became_peered":"2024-04-25T00:55:39.639372+0000","last_unstale":"2024-04-25T00:55:41.021086+0000","last_undegraded":"2024-04-25T00:55:41.021086+0000","last_fullsized":"2024-04-25T00:55:41.021086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636950+0000","last_change":"2024-04-25T00:55:38.613079+0000","last_active":"2024-04-25T00:55:40.636950+0000","last_peered":"2024-04-25T00:55:40.636950+0000","last_clean":"2024-04-25T00:55:40.636950+0000","last_became_active":"2024-04-25T00:55:38.612643+0000","last_became_peered":"2024-04-25T00:55:38.612643+0000","last_unstale":"2024-04-25T00:55:40.636950+0000","last_undegraded":"2024-04-25T00:55:40.636950+0000","last_fullsized":"2024-04-25T00:55:40.636950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021795+0000","last_change":"2024-04-25T00:55:39.640875+0000","last_active":"2024-04-25T00:55:41.021795+0000","last_peered":"2024-04-25T00:55:41.021795+0000","last_clean":"2024-04-25T00:55:41.021795+0000","last_became_active":"2024-04-25T00:55:39.638658+0000","last_became_peered":"2024-04-25T00:55:39.638658+0000","last_unstale":"2024-04-25T00:55:41.021795+0000","last_undegraded":"2024-04-25T00:55:41.021795+0000","last_fullsized":"2024-04-25T00:55:41.021795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017957+0000","last_change":"2024-04-25T00:55:38.624428+0000","last_active":"2024-04-25T00:55:41.017957+0000","last_peered":"2024-04-25T00:55:41.017957+0000","last_clean":"2024-04-25T00:55:41.017957+0000","last_became_active":"2024-04-25T00:55:38.622736+0000","last_became_peered":"2024-04-25T00:55:38.622736+0000","last_unstale":"2024-04-25T00:55:41.017957+0000","last_undegraded":"2024-04-25T00:55:41.017957+0000","last_fullsized":"2024-04-25T00:55:41.017957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:26.077118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652902+0000","last_change":"2024-04-25T00:55:39.631349+0000","last_active":"2024-04-25T00:55:40.652902+0000","last_peered":"2024-04-25T00:55:40.652902+0000","last_clean":"2024-04-25T00:55:40.652902+0000","last_became_active":"2024-04-25T00:55:39.630507+0000","last_became_peered":"2024-04-25T00:55:39.630507+0000","last_unstale":"2024-04-25T00:55:40.652902+0000","last_undegraded":"2024-04-25T00:55:40.652902+0000","last_fullsized":"2024-04-25T00:55:40.652902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637806+0000","last_change":"2024-04-25T00:55:39.628089+0000","last_active":"2024-04-25T00:55:40.637806+0000","last_peered":"2024-04-25T00:55:40.637806+0000","last_clean":"2024-04-25T00:55:40.637806+0000","last_became_active":"2024-04-25T00:55:39.627070+0000","last_became_peered":"2024-04-25T00:55:39.627070+0000","last_unstale":"2024-04-25T00:55:40.637806+0000","last_undegraded":"2024-04-25T00:55:40.637806+0000","last_fullsized":"2024-04-25T00:55:40.637806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017214+0000","last_change":"2024-04-25T00:55:38.634723+0000","last_active":"2024-04-25T00:55:41.017214+0000","last_peered":"2024-04-25T00:55:41.017214+0000","last_clean":"2024-04-25T00:55:41.017214+0000","last_became_active":"2024-04-25T00:55:38.633867+0000","last_became_peered":"2024-04-25T00:55:38.633867+0000","last_unstale":"2024-04-25T00:55:41.017214+0000","last_undegraded":"2024-04-25T00:55:41.017214+0000","last_fullsized":"2024-04-25T00:55:41.017214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638050+0000","last_change":"2024-04-25T00:55:39.618027+0000","last_active":"2024-04-25T00:55:40.638050+0000","last_peered":"2024-04-25T00:55:40.638050+0000","last_clean":"2024-04-25T00:55:40.638050+0000","last_became_active":"2024-04-25T00:55:39.617211+0000","last_became_peered":"2024-04-25T00:55:39.617211+0000","last_unstale":"2024-04-25T00:55:40.638050+0000","last_undegraded":"2024-04-25T00:55:40.638050+0000","last_fullsized":"2024-04-25T00:55:40.638050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:11.034367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017702+0000","last_change":"2024-04-25T00:55:38.636516+0000","last_active":"2024-04-25T00:55:41.017702+0000","last_peered":"2024-04-25T00:55:41.017702+0000","last_clean":"2024-04-25T00:55:41.017702+0000","last_became_active":"2024-04-25T00:55:38.635662+0000","last_became_peered":"2024-04-25T00:55:38.635662+0000","last_unstale":"2024-04-25T00:55:41.017702+0000","last_undegraded":"2024-04-25T00:55:41.017702+0000","last_fullsized":"2024-04-25T00:55:41.017702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018504+0000","last_change":"2024-04-25T00:55:39.622594+0000","last_active":"2024-04-25T00:55:41.018504+0000","last_peered":"2024-04-25T00:55:41.018504+0000","last_clean":"2024-04-25T00:55:41.018504+0000","last_became_active":"2024-04-25T00:55:39.621768+0000","last_became_peered":"2024-04-25T00:55:39.621768+0000","last_unstale":"2024-04-25T00:55:41.018504+0000","last_undegraded":"2024-04-25T00:55:41.018504+0000","last_fullsized":"2024-04-25T00:55:41.018504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:27.074899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021808+0000","last_change":"2024-04-25T00:55:38.609758+0000","last_active":"2024-04-25T00:55:41.021808+0000","last_peered":"2024-04-25T00:55:41.021808+0000","last_clean":"2024-04-25T00:55:41.021808+0000","last_became_active":"2024-04-25T00:55:38.609401+0000","last_became_peered":"2024-04-25T00:55:38.609401+0000","last_unstale":"2024-04-25T00:55:41.021808+0000","last_undegraded":"2024-04-25T00:55:41.021808+0000","last_fullsized":"2024-04-25T00:55:41.021808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:01.720059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021639+0000","last_change":"2024-04-25T00:55:39.657195+0000","last_active":"2024-04-25T00:55:41.021639+0000","last_peered":"2024-04-25T00:55:41.021639+0000","last_clean":"2024-04-25T00:55:41.021639+0000","last_became_active":"2024-04-25T00:55:39.656921+0000","last_became_peered":"2024-04-25T00:55:39.656921+0000","last_unstale":"2024-04-25T00:55:41.021639+0000","last_undegraded":"2024-04-25T00:55:41.021639+0000","last_fullsized":"2024-04-25T00:55:41.021639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021304+0000","last_change":"2024-04-25T00:55:38.629445+0000","last_active":"2024-04-25T00:55:41.021304+0000","last_peered":"2024-04-25T00:55:41.021304+0000","last_clean":"2024-04-25T00:55:41.021304+0000","last_became_active":"2024-04-25T00:55:38.628997+0000","last_became_peered":"2024-04-25T00:55:38.628997+0000","last_unstale":"2024-04-25T00:55:41.021304+0000","last_undegraded":"2024-04-25T00:55:41.021304+0000","last_fullsized":"2024-04-25T00:55:41.021304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:01.720059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:55:43.667 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:55:43.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa7415eb640 1 Processor -- start 2024-04-25T00:55:43.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa7415eb640 1 -- start start 2024-04-25T00:55:43.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa7415eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c06ca10 0x7fa73c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa7415eb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa73c0add70 0x7fa73c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa7415eb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa73c0af760 con 0x7fa73c06c630 2024-04-25T00:55:43.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa7415eb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa73c148860 con 0x7fa73c0add70 2024-04-25T00:55:43.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa7415eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa73c0b52f0 con 0x7fa73c06ca10 2024-04-25T00:55:43.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73affd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa73c06c630 0x7fa73c1477c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57216/0 (socket says 172.21.15.112:57216) 2024-04-25T00:55:43.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73affd640 1 -- 172.21.15.112:0/3140491192 learned_addr learned my addr 172.21.15.112:0/3140491192 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:43.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73a7fc640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c06ca10 0x7fa73c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73b7fe640 1 --2- 172.21.15.112:0/3140491192 >> v2:172.21.15.187:3300/0 conn(0x7fa73c0add70 0x7fa73c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 785681657 0 0) 0x7fa73c0af760 con 0x7fa73c06c630 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa728005740 con 0x7fa73c06c630 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3496038082 0 0) 0x7fa728005740 con 0x7fa73c06c630 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c06ca10 msgr2=0x7fa73c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c06ca10 0x7fa73c0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 >> v2:172.21.15.187:3300/0 conn(0x7fa73c0add70 msgr2=0x7fa73c0b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 --2- 172.21.15.112:0/3140491192 >> v2:172.21.15.187:3300/0 conn(0x7fa73c0add70 0x7fa73c0b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73c0afa00 con 0x7fa73c06c630 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7fa734001a20 con 0x7fa73c06c630 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 >> v1:172.21.15.112:6789/0 conn(0x7fa73c06c630 legacy=0x7fa73c1477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa728005bd0 0x7fa728005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728006a20 0x7fa728006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c06ca10 0x7fa7280073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c0b52f0 con 0x7fa728005bd0 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa72800dc10 con 0x7fa728006a20 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa72800de10 con 0x7fa73c06ca10 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73affd640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa728005bd0 0x7fa728005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73a7fc640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728006a20 0x7fa728006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73affd640 1 -- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c06ca10 msgr2=0x7fa7280073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73affd640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c06ca10 0x7fa7280073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73affd640 1 -- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728006a20 msgr2=0x7fa728006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73affd640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728006a20 0x7fa728006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73affd640 1 -- 172.21.15.112:0/3140491192 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa728005740 con 0x7fa728005bd0 2024-04-25T00:55:43.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.812+0000 7fa73a7fc640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728006a20 0x7fa728006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:43.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73affd640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa728005bd0 0x7fa728005ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa73400a880 tx=0x7fa73401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15aea079a098e783 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73400a4d0 con 0x7fa728005bd0 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7340ad650 con 0x7fa728005bd0 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa739ffb640 1 -- 172.21.15.112:0/3140491192 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7340ada40 con 0x7fa728005bd0 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- 172.21.15.112:0/3140491192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa728005bd0 msgr2=0x7fa728005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 --2- 172.21.15.112:0/3140491192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa728005bd0 0x7fa728005ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa73400a880 tx=0x7fa73401d380 comp rx=0 tx=0).stop 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73b7fe640 1 -- 172.21.15.112:0/3140491192 reap_dead start 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- 172.21.15.112:0/3140491192 shutdown_connections 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- 172.21.15.112:0/3140491192 >> 172.21.15.112:0/3140491192 conn(0x7fa73c0b5980 msgr2=0x7fa73c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- 172.21.15.112:0/3140491192 shutdown_connections 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- 172.21.15.112:0/3140491192 wait complete. 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 Processor -- start 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- start start 2024-04-25T00:55:43.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728005bd0 0x7fa73c149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa73c1db8f0 0x7fa73c13c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c13cf00 0x7fa73c0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c0b3810 con 0x7fa73c1db8f0 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa73c0b3690 con 0x7fa728005bd0 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa7415eb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa73c0b3990 con 0x7fa73c13cf00 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa73c1db8f0 0x7fa73c13c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73a7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa73c1db8f0 0x7fa73c13c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51022/0 (socket says 172.21.15.112:51022) 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73a7fc640 1 -- 172.21.15.112:0/3158118045 learned_addr learned my addr 172.21.15.112:0/3158118045 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73b7fe640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c13cf00 0x7fa73c0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73a7fc640 1 -- 172.21.15.112:0/3158118045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c13cf00 msgr2=0x7fa73c0719e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73a7fc640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c13cf00 0x7fa73c0719e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73affd640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728005bd0 0x7fa73c149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73a7fc640 1 -- 172.21.15.112:0/3158118045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728005bd0 msgr2=0x7fa73c149060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73a7fc640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728005bd0 0x7fa73c149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.816+0000 7fa73a7fc640 1 -- 172.21.15.112:0/3158118045 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa73c13f6c0 con 0x7fa73c1db8f0 2024-04-25T00:55:43.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa73a7fc640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa73c1db8f0 0x7fa73c13c9c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa7240108d0 tx=0x7fa724010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7337fe640 1 -- 172.21.15.112:0/3158118045 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72400f040 con 0x7fa73c1db8f0 2024-04-25T00:55:43.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7337fe640 1 -- 172.21.15.112:0/3158118045 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa724004220 con 0x7fa73c1db8f0 2024-04-25T00:55:43.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa73c1dbe80 con 0x7fa73c1db8f0 2024-04-25T00:55:43.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7337fe640 1 -- 172.21.15.112:0/3158118045 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72400e960 con 0x7fa73c1db8f0 2024-04-25T00:55:43.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa73c1dc440 con 0x7fa73c1db8f0 2024-04-25T00:55:43.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73c06d090 con 0x7fa73c1db8f0 2024-04-25T00:55:43.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7337fe640 1 -- 172.21.15.112:0/3158118045 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa72400eb00 con 0x7fa73c1db8f0 2024-04-25T00:55:43.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7337fe640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa71c0678c0 0x7fa71c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:43.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa7337fe640 1 -- 172.21.15.112:0/3158118045 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa724086490 con 0x7fa73c1db8f0 2024-04-25T00:55:43.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa73affd640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa71c0678c0 0x7fa71c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:43.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.820+0000 7fa73affd640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa71c0678c0 0x7fa71c069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7340ad000 tx=0x7fa734001df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:43.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.824+0000 7fa7337fe640 1 -- 172.21.15.112:0/3158118045 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa724018030 con 0x7fa73c1db8f0 2024-04-25T00:55:43.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.968+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa73c0b0b50 con 0x7fa71c0678c0 2024-04-25T00:55:43.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.980+0000 7fa7337fe640 1 -- 172.21.15.112:0/3158118045 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7fa73c0b0b50 con 0x7fa71c0678c0 2024-04-25T00:55:43.988 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:43.989 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:55:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa71c0678c0 msgr2=0x7fa71c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa71c0678c0 0x7fa71c069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa7340ad000 tx=0x7fa734001df0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa73c1db8f0 msgr2=0x7fa73c13c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa73c1db8f0 0x7fa73c13c9c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa7240108d0 tx=0x7fa724010d90 comp rx=0 tx=0).stop 2024-04-25T00:55:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 shutdown_connections 2024-04-25T00:55:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa73c13cf00 0x7fa73c0719e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa73c1db8f0 0x7fa73c13c9c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa71c0678c0 0x7fa71c069d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 --2- 172.21.15.112:0/3158118045 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa728005bd0 0x7fa73c149060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:43.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 >> 172.21.15.112:0/3158118045 conn(0x7fa73c0b5980 msgr2=0x7fa73c1448b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:43.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 shutdown_connections 2024-04-25T00:55:43.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:43.984+0000 7fa7415eb640 1 -- 172.21.15.112:0/3158118045 wait complete. 2024-04-25T00:55:44.002 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T00:55:43.514219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.935199"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018125+0000","last_change":"2024-04-25T00:55:39.661241+0000","last_active":"2024-04-25T00:55:41.018125+0000","last_peered":"2024-04-25T00:55:41.018125+0000","last_clean":"2024-04-25T00:55:41.018125+0000","last_became_active":"2024-04-25T00:55:39.660150+0000","last_became_peered":"2024-04-25T00:55:39.660150+0000","last_unstale":"2024-04-25T00:55:41.018125+0000","last_undegraded":"2024-04-25T00:55:41.018125+0000","last_fullsized":"2024-04-25T00:55:41.018125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:11.034367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637050+0000","last_change":"2024-04-25T00:55:39.633699+0000","last_active":"2024-04-25T00:55:40.637050+0000","last_peered":"2024-04-25T00:55:40.637050+0000","last_clean":"2024-04-25T00:55:40.637050+0000","last_became_active":"2024-04-25T00:55:39.632942+0000","last_became_peered":"2024-04-25T00:55:39.632942+0000","last_unstale":"2024-04-25T00:55:40.637050+0000","last_undegraded":"2024-04-25T00:55:40.637050+0000","last_fullsized":"2024-04-25T00:55:40.637050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018670+0000","last_change":"2024-04-25T00:55:39.629762+0000","last_active":"2024-04-25T00:55:41.018670+0000","last_peered":"2024-04-25T00:55:41.018670+0000","last_clean":"2024-04-25T00:55:41.018670+0000","last_became_active":"2024-04-25T00:55:39.628798+0000","last_became_peered":"2024-04-25T00:55:39.628798+0000","last_unstale":"2024-04-25T00:55:41.018670+0000","last_undegraded":"2024-04-25T00:55:41.018670+0000","last_fullsized":"2024-04-25T00:55:41.018670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:23.939072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637845+0000","last_change":"2024-04-25T00:55:39.643398+0000","last_active":"2024-04-25T00:55:40.637845+0000","last_peered":"2024-04-25T00:55:40.637845+0000","last_clean":"2024-04-25T00:55:40.637845+0000","last_became_active":"2024-04-25T00:55:39.642844+0000","last_became_peered":"2024-04-25T00:55:39.642844+0000","last_unstale":"2024-04-25T00:55:40.637845+0000","last_undegraded":"2024-04-25T00:55:40.637845+0000","last_fullsized":"2024-04-25T00:55:40.637845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017483+0000","last_change":"2024-04-25T00:55:39.649899+0000","last_active":"2024-04-25T00:55:41.017483+0000","last_peered":"2024-04-25T00:55:41.017483+0000","last_clean":"2024-04-25T00:55:41.017483+0000","last_became_active":"2024-04-25T00:55:39.648852+0000","last_became_peered":"2024-04-25T00:55:39.648852+0000","last_unstale":"2024-04-25T00:55:41.017483+0000","last_undegraded":"2024-04-25T00:55:41.017483+0000","last_fullsized":"2024-04-25T00:55:41.017483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638862+0000","last_change":"2024-04-25T00:55:39.635369+0000","last_active":"2024-04-25T00:55:40.638862+0000","last_peered":"2024-04-25T00:55:40.638862+0000","last_clean":"2024-04-25T00:55:40.638862+0000","last_became_active":"2024-04-25T00:55:39.634678+0000","last_became_peered":"2024-04-25T00:55:39.634678+0000","last_unstale":"2024-04-25T00:55:40.638862+0000","last_undegraded":"2024-04-25T00:55:40.638862+0000","last_fullsized":"2024-04-25T00:55:40.638862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:30.556421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016962+0000","last_change":"2024-04-25T00:55:39.646309+0000","last_active":"2024-04-25T00:55:41.016962+0000","last_peered":"2024-04-25T00:55:41.016962+0000","last_clean":"2024-04-25T00:55:41.016962+0000","last_became_active":"2024-04-25T00:55:39.645778+0000","last_became_peered":"2024-04-25T00:55:39.645778+0000","last_unstale":"2024-04-25T00:55:41.016962+0000","last_undegraded":"2024-04-25T00:55:41.016962+0000","last_fullsized":"2024-04-25T00:55:41.016962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020590+0000","last_change":"2024-04-25T00:55:39.643250+0000","last_active":"2024-04-25T00:55:41.020590+0000","last_peered":"2024-04-25T00:55:41.020590+0000","last_clean":"2024-04-25T00:55:41.020590+0000","last_became_active":"2024-04-25T00:55:39.642948+0000","last_became_peered":"2024-04-25T00:55:39.642948+0000","last_unstale":"2024-04-25T00:55:41.020590+0000","last_undegraded":"2024-04-25T00:55:41.020590+0000","last_fullsized":"2024-04-25T00:55:41.020590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:53.762656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018212+0000","last_change":"2024-04-25T00:55:39.658665+0000","last_active":"2024-04-25T00:55:41.018212+0000","last_peered":"2024-04-25T00:55:41.018212+0000","last_clean":"2024-04-25T00:55:41.018212+0000","last_became_active":"2024-04-25T00:55:39.658276+0000","last_became_peered":"2024-04-25T00:55:39.658276+0000","last_unstale":"2024-04-25T00:55:41.018212+0000","last_undegraded":"2024-04-25T00:55:41.018212+0000","last_fullsized":"2024-04-25T00:55:41.018212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:47.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652697+0000","last_change":"2024-04-25T00:55:39.660203+0000","last_active":"2024-04-25T00:55:40.652697+0000","last_peered":"2024-04-25T00:55:40.652697+0000","last_clean":"2024-04-25T00:55:40.652697+0000","last_became_active":"2024-04-25T00:55:39.656617+0000","last_became_peered":"2024-04-25T00:55:39.656617+0000","last_unstale":"2024-04-25T00:55:40.652697+0000","last_undegraded":"2024-04-25T00:55:40.652697+0000","last_fullsized":"2024-04-25T00:55:40.652697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652875+0000","last_change":"2024-04-25T00:55:39.638495+0000","last_active":"2024-04-25T00:55:40.652875+0000","last_peered":"2024-04-25T00:55:40.652875+0000","last_clean":"2024-04-25T00:55:40.652875+0000","last_became_active":"2024-04-25T00:55:39.631516+0000","last_became_peered":"2024-04-25T00:55:39.631516+0000","last_unstale":"2024-04-25T00:55:40.652875+0000","last_undegraded":"2024-04-25T00:55:40.652875+0000","last_fullsized":"2024-04-25T00:55:40.652875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636317+0000","last_change":"2024-04-25T00:55:39.657591+0000","last_active":"2024-04-25T00:55:40.636317+0000","last_peered":"2024-04-25T00:55:40.636317+0000","last_clean":"2024-04-25T00:55:40.636317+0000","last_became_active":"2024-04-25T00:55:39.657259+0000","last_became_peered":"2024-04-25T00:55:39.657259+0000","last_unstale":"2024-04-25T00:55:40.636317+0000","last_undegraded":"2024-04-25T00:55:40.636317+0000","last_fullsized":"2024-04-25T00:55:40.636317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:56.786582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.022200+0000","last_change":"2024-04-25T00:55:39.656495+0000","last_active":"2024-04-25T00:55:41.022200+0000","last_peered":"2024-04-25T00:55:41.022200+0000","last_clean":"2024-04-25T00:55:41.022200+0000","last_became_active":"2024-04-25T00:55:39.655994+0000","last_became_peered":"2024-04-25T00:55:39.655994+0000","last_unstale":"2024-04-25T00:55:41.022200+0000","last_undegraded":"2024-04-25T00:55:41.022200+0000","last_fullsized":"2024-04-25T00:55:41.022200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019282+0000","last_change":"2024-04-25T00:55:39.636352+0000","last_active":"2024-04-25T00:55:41.019282+0000","last_peered":"2024-04-25T00:55:41.019282+0000","last_clean":"2024-04-25T00:55:41.019282+0000","last_became_active":"2024-04-25T00:55:39.636013+0000","last_became_peered":"2024-04-25T00:55:39.636013+0000","last_unstale":"2024-04-25T00:55:41.019282+0000","last_undegraded":"2024-04-25T00:55:41.019282+0000","last_fullsized":"2024-04-25T00:55:41.019282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636741+0000","last_change":"2024-04-25T00:55:39.634943+0000","last_active":"2024-04-25T00:55:40.636741+0000","last_peered":"2024-04-25T00:55:40.636741+0000","last_clean":"2024-04-25T00:55:40.636741+0000","last_became_active":"2024-04-25T00:55:39.634116+0000","last_became_peered":"2024-04-25T00:55:39.634116+0000","last_unstale":"2024-04-25T00:55:40.636741+0000","last_undegraded":"2024-04-25T00:55:40.636741+0000","last_fullsized":"2024-04-25T00:55:40.636741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637210+0000","last_change":"2024-04-25T00:55:39.630149+0000","last_active":"2024-04-25T00:55:40.637210+0000","last_peered":"2024-04-25T00:55:40.637210+0000","last_clean":"2024-04-25T00:55:40.637210+0000","last_became_active":"2024-04-25T00:55:39.629387+0000","last_became_peered":"2024-04-25T00:55:39.629387+0000","last_unstale":"2024-04-25T00:55:40.637210+0000","last_undegraded":"2024-04-25T00:55:40.637210+0000","last_fullsized":"2024-04-25T00:55:40.637210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638541+0000","last_change":"2024-04-25T00:55:39.650995+0000","last_active":"2024-04-25T00:55:40.638541+0000","last_peered":"2024-04-25T00:55:40.638541+0000","last_clean":"2024-04-25T00:55:40.638541+0000","last_became_active":"2024-04-25T00:55:39.650711+0000","last_became_peered":"2024-04-25T00:55:39.650711+0000","last_unstale":"2024-04-25T00:55:40.638541+0000","last_undegraded":"2024-04-25T00:55:40.638541+0000","last_fullsized":"2024-04-25T00:55:40.638541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.187193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019011+0000","last_change":"2024-04-25T00:55:39.656754+0000","last_active":"2024-04-25T00:55:41.019011+0000","last_peered":"2024-04-25T00:55:41.019011+0000","last_clean":"2024-04-25T00:55:41.019011+0000","last_became_active":"2024-04-25T00:55:39.656397+0000","last_became_peered":"2024-04-25T00:55:39.656397+0000","last_unstale":"2024-04-25T00:55:41.019011+0000","last_undegraded":"2024-04-25T00:55:41.019011+0000","last_fullsized":"2024-04-25T00:55:41.019011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.087648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018205+0000","last_change":"2024-04-25T00:55:39.648560+0000","last_active":"2024-04-25T00:55:41.018205+0000","last_peered":"2024-04-25T00:55:41.018205+0000","last_clean":"2024-04-25T00:55:41.018205+0000","last_became_active":"2024-04-25T00:55:39.647080+0000","last_became_peered":"2024-04-25T00:55:39.647080+0000","last_unstale":"2024-04-25T00:55:41.018205+0000","last_undegraded":"2024-04-25T00:55:41.018205+0000","last_fullsized":"2024-04-25T00:55:41.018205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:22.249474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.653020+0000","last_change":"2024-04-25T00:55:39.659058+0000","last_active":"2024-04-25T00:55:40.653020+0000","last_peered":"2024-04-25T00:55:40.653020+0000","last_clean":"2024-04-25T00:55:40.653020+0000","last_became_active":"2024-04-25T00:55:39.657152+0000","last_became_peered":"2024-04-25T00:55:39.657152+0000","last_unstale":"2024-04-25T00:55:40.653020+0000","last_undegraded":"2024-04-25T00:55:40.653020+0000","last_fullsized":"2024-04-25T00:55:40.653020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020095+0000","last_change":"2024-04-25T00:55:39.650890+0000","last_active":"2024-04-25T00:55:41.020095+0000","last_peered":"2024-04-25T00:55:41.020095+0000","last_clean":"2024-04-25T00:55:41.020095+0000","last_became_active":"2024-04-25T00:55:39.650235+0000","last_became_peered":"2024-04-25T00:55:39.650235+0000","last_unstale":"2024-04-25T00:55:41.020095+0000","last_undegraded":"2024-04-25T00:55:41.020095+0000","last_fullsized":"2024-04-25T00:55:41.020095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638004+0000","last_change":"2024-04-25T00:55:39.646725+0000","last_active":"2024-04-25T00:55:40.638004+0000","last_peered":"2024-04-25T00:55:40.638004+0000","last_clean":"2024-04-25T00:55:40.638004+0000","last_became_active":"2024-04-25T00:55:39.646192+0000","last_became_peered":"2024-04-25T00:55:39.646192+0000","last_unstale":"2024-04-25T00:55:40.638004+0000","last_undegraded":"2024-04-25T00:55:40.638004+0000","last_fullsized":"2024-04-25T00:55:40.638004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:51.008333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020594+0000","last_change":"2024-04-25T00:55:39.642303+0000","last_active":"2024-04-25T00:55:41.020594+0000","last_peered":"2024-04-25T00:55:41.020594+0000","last_clean":"2024-04-25T00:55:41.020594+0000","last_became_active":"2024-04-25T00:55:39.641627+0000","last_became_peered":"2024-04-25T00:55:39.641627+0000","last_unstale":"2024-04-25T00:55:41.020594+0000","last_undegraded":"2024-04-25T00:55:41.020594+0000","last_fullsized":"2024-04-25T00:55:41.020594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637271+0000","last_change":"2024-04-25T00:55:39.643919+0000","last_active":"2024-04-25T00:55:40.637271+0000","last_peered":"2024-04-25T00:55:40.637271+0000","last_clean":"2024-04-25T00:55:40.637271+0000","last_became_active":"2024-04-25T00:55:39.643196+0000","last_became_peered":"2024-04-25T00:55:39.643196+0000","last_unstale":"2024-04-25T00:55:40.637271+0000","last_undegraded":"2024-04-25T00:55:40.637271+0000","last_fullsized":"2024-04-25T00:55:40.637271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021581+0000","last_change":"2024-04-25T00:55:39.641462+0000","last_active":"2024-04-25T00:55:41.021581+0000","last_peered":"2024-04-25T00:55:41.021581+0000","last_clean":"2024-04-25T00:55:41.021581+0000","last_became_active":"2024-04-25T00:55:39.640648+0000","last_became_peered":"2024-04-25T00:55:39.640648+0000","last_unstale":"2024-04-25T00:55:41.021581+0000","last_undegraded":"2024-04-25T00:55:41.021581+0000","last_fullsized":"2024-04-25T00:55:41.021581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652767+0000","last_change":"2024-04-25T00:55:39.648816+0000","last_active":"2024-04-25T00:55:40.652767+0000","last_peered":"2024-04-25T00:55:40.652767+0000","last_clean":"2024-04-25T00:55:40.652767+0000","last_became_active":"2024-04-25T00:55:39.648337+0000","last_became_peered":"2024-04-25T00:55:39.648337+0000","last_unstale":"2024-04-25T00:55:40.652767+0000","last_undegraded":"2024-04-25T00:55:40.652767+0000","last_fullsized":"2024-04-25T00:55:40.652767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637430+0000","last_change":"2024-04-25T00:55:39.644333+0000","last_active":"2024-04-25T00:55:40.637430+0000","last_peered":"2024-04-25T00:55:40.637430+0000","last_clean":"2024-04-25T00:55:40.637430+0000","last_became_active":"2024-04-25T00:55:39.643706+0000","last_became_peered":"2024-04-25T00:55:39.643706+0000","last_unstale":"2024-04-25T00:55:40.637430+0000","last_undegraded":"2024-04-25T00:55:40.637430+0000","last_fullsized":"2024-04-25T00:55:40.637430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021853+0000","last_change":"2024-04-25T00:55:39.631560+0000","last_active":"2024-04-25T00:55:41.021853+0000","last_peered":"2024-04-25T00:55:41.021853+0000","last_clean":"2024-04-25T00:55:41.021853+0000","last_became_active":"2024-04-25T00:55:39.630796+0000","last_became_peered":"2024-04-25T00:55:39.630796+0000","last_unstale":"2024-04-25T00:55:41.021853+0000","last_undegraded":"2024-04-25T00:55:41.021853+0000","last_fullsized":"2024-04-25T00:55:41.021853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020172+0000","last_change":"2024-04-25T00:55:39.643191+0000","last_active":"2024-04-25T00:55:41.020172+0000","last_peered":"2024-04-25T00:55:41.020172+0000","last_clean":"2024-04-25T00:55:41.020172+0000","last_became_active":"2024-04-25T00:55:39.642846+0000","last_became_peered":"2024-04-25T00:55:39.642846+0000","last_unstale":"2024-04-25T00:55:41.020172+0000","last_undegraded":"2024-04-25T00:55:41.020172+0000","last_fullsized":"2024-04-25T00:55:41.020172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638416+0000","last_change":"2024-04-25T00:55:39.643174+0000","last_active":"2024-04-25T00:55:40.638416+0000","last_peered":"2024-04-25T00:55:40.638416+0000","last_clean":"2024-04-25T00:55:40.638416+0000","last_became_active":"2024-04-25T00:55:39.642575+0000","last_became_peered":"2024-04-25T00:55:39.642575+0000","last_unstale":"2024-04-25T00:55:40.638416+0000","last_undegraded":"2024-04-25T00:55:40.638416+0000","last_fullsized":"2024-04-25T00:55:40.638416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:47.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018168+0000","last_change":"2024-04-25T00:55:39.627384+0000","last_active":"2024-04-25T00:55:41.018168+0000","last_peered":"2024-04-25T00:55:41.018168+0000","last_clean":"2024-04-25T00:55:41.018168+0000","last_became_active":"2024-04-25T00:55:39.627078+0000","last_became_peered":"2024-04-25T00:55:39.627078+0000","last_unstale":"2024-04-25T00:55:41.018168+0000","last_undegraded":"2024-04-25T00:55:41.018168+0000","last_fullsized":"2024-04-25T00:55:41.018168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:21.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652573+0000","last_change":"2024-04-25T00:55:39.655121+0000","last_active":"2024-04-25T00:55:40.652573+0000","last_peered":"2024-04-25T00:55:40.652573+0000","last_clean":"2024-04-25T00:55:40.652573+0000","last_became_active":"2024-04-25T00:55:39.654774+0000","last_became_peered":"2024-04-25T00:55:39.654774+0000","last_unstale":"2024-04-25T00:55:40.652573+0000","last_undegraded":"2024-04-25T00:55:40.652573+0000","last_fullsized":"2024-04-25T00:55:40.652573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636316+0000","last_change":"2024-04-25T00:55:39.634737+0000","last_active":"2024-04-25T00:55:40.636316+0000","last_peered":"2024-04-25T00:55:40.636316+0000","last_clean":"2024-04-25T00:55:40.636316+0000","last_became_active":"2024-04-25T00:55:39.633943+0000","last_became_peered":"2024-04-25T00:55:39.633943+0000","last_unstale":"2024-04-25T00:55:40.636316+0000","last_undegraded":"2024-04-25T00:55:40.636316+0000","last_fullsized":"2024-04-25T00:55:40.636316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:09.262927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021945+0000","last_change":"2024-04-25T00:55:38.629602+0000","last_active":"2024-04-25T00:55:41.021945+0000","last_peered":"2024-04-25T00:55:41.021945+0000","last_clean":"2024-04-25T00:55:41.021945+0000","last_became_active":"2024-04-25T00:55:38.628956+0000","last_became_peered":"2024-04-25T00:55:38.628956+0000","last_unstale":"2024-04-25T00:55:41.021945+0000","last_undegraded":"2024-04-25T00:55:41.021945+0000","last_fullsized":"2024-04-25T00:55:41.021945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:50.369672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020489+0000","last_change":"2024-04-25T00:55:39.630640+0000","last_active":"2024-04-25T00:55:41.020489+0000","last_peered":"2024-04-25T00:55:41.020489+0000","last_clean":"2024-04-25T00:55:41.020489+0000","last_became_active":"2024-04-25T00:55:39.629906+0000","last_became_peered":"2024-04-25T00:55:39.629906+0000","last_unstale":"2024-04-25T00:55:41.020489+0000","last_undegraded":"2024-04-25T00:55:41.020489+0000","last_fullsized":"2024-04-25T00:55:41.020489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652654+0000","last_change":"2024-04-25T00:55:38.626215+0000","last_active":"2024-04-25T00:55:40.652654+0000","last_peered":"2024-04-25T00:55:40.652654+0000","last_clean":"2024-04-25T00:55:40.652654+0000","last_became_active":"2024-04-25T00:55:38.625439+0000","last_became_peered":"2024-04-25T00:55:38.625439+0000","last_unstale":"2024-04-25T00:55:40.652654+0000","last_undegraded":"2024-04-25T00:55:40.652654+0000","last_fullsized":"2024-04-25T00:55:40.652654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020655+0000","last_change":"2024-04-25T00:55:38.626401+0000","last_active":"2024-04-25T00:55:41.020655+0000","last_peered":"2024-04-25T00:55:41.020655+0000","last_clean":"2024-04-25T00:55:41.020655+0000","last_became_active":"2024-04-25T00:55:38.625884+0000","last_became_peered":"2024-04-25T00:55:38.625884+0000","last_unstale":"2024-04-25T00:55:41.020655+0000","last_undegraded":"2024-04-25T00:55:41.020655+0000","last_fullsized":"2024-04-25T00:55:41.020655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020981+0000","last_change":"2024-04-25T00:55:39.638181+0000","last_active":"2024-04-25T00:55:41.020981+0000","last_peered":"2024-04-25T00:55:41.020981+0000","last_clean":"2024-04-25T00:55:41.020981+0000","last_became_active":"2024-04-25T00:55:39.637900+0000","last_became_peered":"2024-04-25T00:55:39.637900+0000","last_unstale":"2024-04-25T00:55:41.020981+0000","last_undegraded":"2024-04-25T00:55:41.020981+0000","last_fullsized":"2024-04-25T00:55:41.020981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637316+0000","last_change":"2024-04-25T00:55:38.621335+0000","last_active":"2024-04-25T00:55:40.637316+0000","last_peered":"2024-04-25T00:55:40.637316+0000","last_clean":"2024-04-25T00:55:40.637316+0000","last_became_active":"2024-04-25T00:55:38.620467+0000","last_became_peered":"2024-04-25T00:55:38.620467+0000","last_unstale":"2024-04-25T00:55:40.637316+0000","last_undegraded":"2024-04-25T00:55:40.637316+0000","last_fullsized":"2024-04-25T00:55:40.637316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636806+0000","last_change":"2024-04-25T00:55:39.642780+0000","last_active":"2024-04-25T00:55:40.636806+0000","last_peered":"2024-04-25T00:55:40.636806+0000","last_clean":"2024-04-25T00:55:40.636806+0000","last_became_active":"2024-04-25T00:55:39.642410+0000","last_became_peered":"2024-04-25T00:55:39.642410+0000","last_unstale":"2024-04-25T00:55:40.636806+0000","last_undegraded":"2024-04-25T00:55:40.636806+0000","last_fullsized":"2024-04-25T00:55:40.636806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652622+0000","last_change":"2024-04-25T00:55:38.634413+0000","last_active":"2024-04-25T00:55:40.652622+0000","last_peered":"2024-04-25T00:55:40.652622+0000","last_clean":"2024-04-25T00:55:40.652622+0000","last_became_active":"2024-04-25T00:55:38.633760+0000","last_became_peered":"2024-04-25T00:55:38.633760+0000","last_unstale":"2024-04-25T00:55:40.652622+0000","last_undegraded":"2024-04-25T00:55:40.652622+0000","last_fullsized":"2024-04-25T00:55:40.652622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637854+0000","last_change":"2024-04-25T00:55:39.644911+0000","last_active":"2024-04-25T00:55:40.637854+0000","last_peered":"2024-04-25T00:55:40.637854+0000","last_clean":"2024-04-25T00:55:40.637854+0000","last_became_active":"2024-04-25T00:55:39.644568+0000","last_became_peered":"2024-04-25T00:55:39.644568+0000","last_unstale":"2024-04-25T00:55:40.637854+0000","last_undegraded":"2024-04-25T00:55:40.637854+0000","last_fullsized":"2024-04-25T00:55:40.637854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:16.517320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637925+0000","last_change":"2024-04-25T00:55:38.637548+0000","last_active":"2024-04-25T00:55:40.637925+0000","last_peered":"2024-04-25T00:55:40.637925+0000","last_clean":"2024-04-25T00:55:40.637925+0000","last_became_active":"2024-04-25T00:55:38.636962+0000","last_became_peered":"2024-04-25T00:55:38.636962+0000","last_unstale":"2024-04-25T00:55:40.637925+0000","last_undegraded":"2024-04-25T00:55:40.637925+0000","last_fullsized":"2024-04-25T00:55:40.637925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020866+0000","last_change":"2024-04-25T00:55:39.628306+0000","last_active":"2024-04-25T00:55:41.020866+0000","last_peered":"2024-04-25T00:55:41.020866+0000","last_clean":"2024-04-25T00:55:41.020866+0000","last_became_active":"2024-04-25T00:55:39.627530+0000","last_became_peered":"2024-04-25T00:55:39.627530+0000","last_unstale":"2024-04-25T00:55:41.020866+0000","last_undegraded":"2024-04-25T00:55:41.020866+0000","last_fullsized":"2024-04-25T00:55:41.020866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637335+0000","last_change":"2024-04-25T00:55:38.640365+0000","last_active":"2024-04-25T00:55:40.637335+0000","last_peered":"2024-04-25T00:55:40.637335+0000","last_clean":"2024-04-25T00:55:40.637335+0000","last_became_active":"2024-04-25T00:55:38.639988+0000","last_became_peered":"2024-04-25T00:55:38.639988+0000","last_unstale":"2024-04-25T00:55:40.637335+0000","last_undegraded":"2024-04-25T00:55:40.637335+0000","last_fullsized":"2024-04-25T00:55:40.637335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636444+0000","last_change":"2024-04-25T00:55:39.647034+0000","last_active":"2024-04-25T00:55:40.636444+0000","last_peered":"2024-04-25T00:55:40.636444+0000","last_clean":"2024-04-25T00:55:40.636444+0000","last_became_active":"2024-04-25T00:55:39.644279+0000","last_became_peered":"2024-04-25T00:55:39.644279+0000","last_unstale":"2024-04-25T00:55:40.636444+0000","last_undegraded":"2024-04-25T00:55:40.636444+0000","last_fullsized":"2024-04-25T00:55:40.636444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:21.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652840+0000","last_change":"2024-04-25T00:55:38.626291+0000","last_active":"2024-04-25T00:55:40.652840+0000","last_peered":"2024-04-25T00:55:40.652840+0000","last_clean":"2024-04-25T00:55:40.652840+0000","last_became_active":"2024-04-25T00:55:38.625530+0000","last_became_peered":"2024-04-25T00:55:38.625530+0000","last_unstale":"2024-04-25T00:55:40.652840+0000","last_undegraded":"2024-04-25T00:55:40.652840+0000","last_fullsized":"2024-04-25T00:55:40.652840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017484+0000","last_change":"2024-04-25T00:55:39.649518+0000","last_active":"2024-04-25T00:55:41.017484+0000","last_peered":"2024-04-25T00:55:41.017484+0000","last_clean":"2024-04-25T00:55:41.017484+0000","last_became_active":"2024-04-25T00:55:39.648701+0000","last_became_peered":"2024-04-25T00:55:39.648701+0000","last_unstale":"2024-04-25T00:55:41.017484+0000","last_undegraded":"2024-04-25T00:55:41.017484+0000","last_fullsized":"2024-04-25T00:55:41.017484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.552805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020445+0000","last_change":"2024-04-25T00:55:38.610242+0000","last_active":"2024-04-25T00:55:41.020445+0000","last_peered":"2024-04-25T00:55:41.020445+0000","last_clean":"2024-04-25T00:55:41.020445+0000","last_became_active":"2024-04-25T00:55:38.609855+0000","last_became_peered":"2024-04-25T00:55:38.609855+0000","last_unstale":"2024-04-25T00:55:41.020445+0000","last_undegraded":"2024-04-25T00:55:41.020445+0000","last_fullsized":"2024-04-25T00:55:41.020445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019689+0000","last_change":"2024-04-25T00:55:39.646421+0000","last_active":"2024-04-25T00:55:41.019689+0000","last_peered":"2024-04-25T00:55:41.019689+0000","last_clean":"2024-04-25T00:55:41.019689+0000","last_became_active":"2024-04-25T00:55:39.646119+0000","last_became_peered":"2024-04-25T00:55:39.646119+0000","last_unstale":"2024-04-25T00:55:41.019689+0000","last_undegraded":"2024-04-25T00:55:41.019689+0000","last_fullsized":"2024-04-25T00:55:41.019689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638593+0000","last_change":"2024-04-25T00:55:38.632875+0000","last_active":"2024-04-25T00:55:40.638593+0000","last_peered":"2024-04-25T00:55:40.638593+0000","last_clean":"2024-04-25T00:55:40.638593+0000","last_became_active":"2024-04-25T00:55:38.631370+0000","last_became_peered":"2024-04-25T00:55:38.631370+0000","last_unstale":"2024-04-25T00:55:40.638593+0000","last_undegraded":"2024-04-25T00:55:40.638593+0000","last_fullsized":"2024-04-25T00:55:40.638593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017573+0000","last_change":"2024-04-25T00:55:39.661000+0000","last_active":"2024-04-25T00:55:41.017573+0000","last_peered":"2024-04-25T00:55:41.017573+0000","last_clean":"2024-04-25T00:55:41.017573+0000","last_became_active":"2024-04-25T00:55:39.660704+0000","last_became_peered":"2024-04-25T00:55:39.660704+0000","last_unstale":"2024-04-25T00:55:41.017573+0000","last_undegraded":"2024-04-25T00:55:41.017573+0000","last_fullsized":"2024-04-25T00:55:41.017573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:16.517320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652441+0000","last_change":"2024-04-25T00:55:38.614039+0000","last_active":"2024-04-25T00:55:40.652441+0000","last_peered":"2024-04-25T00:55:40.652441+0000","last_clean":"2024-04-25T00:55:40.652441+0000","last_became_active":"2024-04-25T00:55:38.612513+0000","last_became_peered":"2024-04-25T00:55:38.612513+0000","last_unstale":"2024-04-25T00:55:40.652441+0000","last_undegraded":"2024-04-25T00:55:40.652441+0000","last_fullsized":"2024-04-25T00:55:40.652441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652439+0000","last_change":"2024-04-25T00:55:39.639974+0000","last_active":"2024-04-25T00:55:40.652439+0000","last_peered":"2024-04-25T00:55:40.652439+0000","last_clean":"2024-04-25T00:55:40.652439+0000","last_became_active":"2024-04-25T00:55:39.639397+0000","last_became_peered":"2024-04-25T00:55:39.639397+0000","last_unstale":"2024-04-25T00:55:40.652439+0000","last_undegraded":"2024-04-25T00:55:40.652439+0000","last_fullsized":"2024-04-25T00:55:40.652439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017966+0000","last_change":"2024-04-25T00:55:38.601709+0000","last_active":"2024-04-25T00:55:41.017966+0000","last_peered":"2024-04-25T00:55:41.017966+0000","last_clean":"2024-04-25T00:55:41.017966+0000","last_became_active":"2024-04-25T00:55:38.601079+0000","last_became_peered":"2024-04-25T00:55:38.601079+0000","last_unstale":"2024-04-25T00:55:41.017966+0000","last_undegraded":"2024-04-25T00:55:41.017966+0000","last_fullsized":"2024-04-25T00:55:41.017966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:50.369672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016947+0000","last_change":"2024-04-25T00:55:39.652284+0000","last_active":"2024-04-25T00:55:41.016947+0000","last_peered":"2024-04-25T00:55:41.016947+0000","last_clean":"2024-04-25T00:55:41.016947+0000","last_became_active":"2024-04-25T00:55:39.651442+0000","last_became_peered":"2024-04-25T00:55:39.651442+0000","last_unstale":"2024-04-25T00:55:41.016947+0000","last_undegraded":"2024-04-25T00:55:41.016947+0000","last_fullsized":"2024-04-25T00:55:41.016947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636948+0000","last_change":"2024-04-25T00:55:38.603875+0000","last_active":"2024-04-25T00:55:40.636948+0000","last_peered":"2024-04-25T00:55:40.636948+0000","last_clean":"2024-04-25T00:55:40.636948+0000","last_became_active":"2024-04-25T00:55:38.602412+0000","last_became_peered":"2024-04-25T00:55:38.602412+0000","last_unstale":"2024-04-25T00:55:40.636948+0000","last_undegraded":"2024-04-25T00:55:40.636948+0000","last_fullsized":"2024-04-25T00:55:40.636948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:34.488773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018026+0000","last_change":"2024-04-25T00:55:39.651592+0000","last_active":"2024-04-25T00:55:41.018026+0000","last_peered":"2024-04-25T00:55:41.018026+0000","last_clean":"2024-04-25T00:55:41.018026+0000","last_became_active":"2024-04-25T00:55:39.651045+0000","last_became_peered":"2024-04-25T00:55:39.651045+0000","last_unstale":"2024-04-25T00:55:41.018026+0000","last_undegraded":"2024-04-25T00:55:41.018026+0000","last_fullsized":"2024-04-25T00:55:41.018026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:51.008333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018836+0000","last_change":"2024-04-25T00:55:38.613063+0000","last_active":"2024-04-25T00:55:41.018836+0000","last_peered":"2024-04-25T00:55:41.018836+0000","last_clean":"2024-04-25T00:55:41.018836+0000","last_became_active":"2024-04-25T00:55:38.612651+0000","last_became_peered":"2024-04-25T00:55:38.612651+0000","last_unstale":"2024-04-25T00:55:41.018836+0000","last_undegraded":"2024-04-25T00:55:41.018836+0000","last_fullsized":"2024-04-25T00:55:41.018836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:25.842602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018376+0000","last_change":"2024-04-25T00:55:39.661043+0000","last_active":"2024-04-25T00:55:41.018376+0000","last_peered":"2024-04-25T00:55:41.018376+0000","last_clean":"2024-04-25T00:55:41.018376+0000","last_became_active":"2024-04-25T00:55:39.660577+0000","last_became_peered":"2024-04-25T00:55:39.660577+0000","last_unstale":"2024-04-25T00:55:41.018376+0000","last_undegraded":"2024-04-25T00:55:41.018376+0000","last_fullsized":"2024-04-25T00:55:41.018376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.187193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637626+0000","last_change":"2024-04-25T00:55:38.610833+0000","last_active":"2024-04-25T00:55:40.637626+0000","last_peered":"2024-04-25T00:55:40.637626+0000","last_clean":"2024-04-25T00:55:40.637626+0000","last_became_active":"2024-04-25T00:55:38.610476+0000","last_became_peered":"2024-04-25T00:55:38.610476+0000","last_unstale":"2024-04-25T00:55:40.637626+0000","last_undegraded":"2024-04-25T00:55:40.637626+0000","last_fullsized":"2024-04-25T00:55:40.637626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638296+0000","last_change":"2024-04-25T00:55:39.643691+0000","last_active":"2024-04-25T00:55:40.638296+0000","last_peered":"2024-04-25T00:55:40.638296+0000","last_clean":"2024-04-25T00:55:40.638296+0000","last_became_active":"2024-04-25T00:55:39.643024+0000","last_became_peered":"2024-04-25T00:55:39.643024+0000","last_unstale":"2024-04-25T00:55:40.638296+0000","last_undegraded":"2024-04-25T00:55:40.638296+0000","last_fullsized":"2024-04-25T00:55:40.638296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:22.249474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020063+0000","last_change":"2024-04-25T00:55:38.623842+0000","last_active":"2024-04-25T00:55:41.020063+0000","last_peered":"2024-04-25T00:55:41.020063+0000","last_clean":"2024-04-25T00:55:41.020063+0000","last_became_active":"2024-04-25T00:55:38.623514+0000","last_became_peered":"2024-04-25T00:55:38.623514+0000","last_unstale":"2024-04-25T00:55:41.020063+0000","last_undegraded":"2024-04-25T00:55:41.020063+0000","last_fullsized":"2024-04-25T00:55:41.020063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017354+0000","last_change":"2024-04-25T00:55:39.658238+0000","last_active":"2024-04-25T00:55:41.017354+0000","last_peered":"2024-04-25T00:55:41.017354+0000","last_clean":"2024-04-25T00:55:41.017354+0000","last_became_active":"2024-04-25T00:55:39.657889+0000","last_became_peered":"2024-04-25T00:55:39.657889+0000","last_unstale":"2024-04-25T00:55:41.017354+0000","last_undegraded":"2024-04-25T00:55:41.017354+0000","last_fullsized":"2024-04-25T00:55:41.017354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016737+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:55:41.016737+0000","last_peered":"2024-04-25T00:55:41.016737+0000","last_clean":"2024-04-25T00:55:41.016737+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:55:41.016737+0000","last_undegraded":"2024-04-25T00:55:41.016737+0000","last_fullsized":"2024-04-25T00:55:41.016737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637451+0000","last_change":"2024-04-25T00:55:38.632700+0000","last_active":"2024-04-25T00:55:40.637451+0000","last_peered":"2024-04-25T00:55:40.637451+0000","last_clean":"2024-04-25T00:55:40.637451+0000","last_became_active":"2024-04-25T00:55:38.631901+0000","last_became_peered":"2024-04-25T00:55:38.631901+0000","last_unstale":"2024-04-25T00:55:40.637451+0000","last_undegraded":"2024-04-25T00:55:40.637451+0000","last_fullsized":"2024-04-25T00:55:40.637451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021271+0000","last_change":"2024-04-25T00:55:39.639013+0000","last_active":"2024-04-25T00:55:41.021271+0000","last_peered":"2024-04-25T00:55:41.021271+0000","last_clean":"2024-04-25T00:55:41.021271+0000","last_became_active":"2024-04-25T00:55:39.638189+0000","last_became_peered":"2024-04-25T00:55:39.638189+0000","last_unstale":"2024-04-25T00:55:41.021271+0000","last_undegraded":"2024-04-25T00:55:41.021271+0000","last_fullsized":"2024-04-25T00:55:41.021271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016724+0000","last_change":"2024-04-25T00:55:38.621598+0000","last_active":"2024-04-25T00:55:41.016724+0000","last_peered":"2024-04-25T00:55:41.016724+0000","last_clean":"2024-04-25T00:55:41.016724+0000","last_became_active":"2024-04-25T00:55:38.620844+0000","last_became_peered":"2024-04-25T00:55:38.620844+0000","last_unstale":"2024-04-25T00:55:41.016724+0000","last_undegraded":"2024-04-25T00:55:41.016724+0000","last_fullsized":"2024-04-25T00:55:41.016724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019793+0000","last_change":"2024-04-25T00:55:39.626650+0000","last_active":"2024-04-25T00:55:41.019793+0000","last_peered":"2024-04-25T00:55:41.019793+0000","last_clean":"2024-04-25T00:55:41.019793+0000","last_became_active":"2024-04-25T00:55:39.625876+0000","last_became_peered":"2024-04-25T00:55:39.625876+0000","last_unstale":"2024-04-25T00:55:41.019793+0000","last_undegraded":"2024-04-25T00:55:41.019793+0000","last_fullsized":"2024-04-25T00:55:41.019793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018447+0000","last_change":"2024-04-25T00:55:38.605319+0000","last_active":"2024-04-25T00:55:41.018447+0000","last_peered":"2024-04-25T00:55:41.018447+0000","last_clean":"2024-04-25T00:55:41.018447+0000","last_became_active":"2024-04-25T00:55:38.604920+0000","last_became_peered":"2024-04-25T00:55:38.604920+0000","last_unstale":"2024-04-25T00:55:41.018447+0000","last_undegraded":"2024-04-25T00:55:41.018447+0000","last_fullsized":"2024-04-25T00:55:41.018447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636431+0000","last_change":"2024-04-25T00:55:39.635799+0000","last_active":"2024-04-25T00:55:40.636431+0000","last_peered":"2024-04-25T00:55:40.636431+0000","last_clean":"2024-04-25T00:55:40.636431+0000","last_became_active":"2024-04-25T00:55:39.633616+0000","last_became_peered":"2024-04-25T00:55:39.633616+0000","last_unstale":"2024-04-25T00:55:40.636431+0000","last_undegraded":"2024-04-25T00:55:40.636431+0000","last_fullsized":"2024-04-25T00:55:40.636431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638712+0000","last_change":"2024-04-25T00:55:38.638178+0000","last_active":"2024-04-25T00:55:40.638712+0000","last_peered":"2024-04-25T00:55:40.638712+0000","last_clean":"2024-04-25T00:55:40.638712+0000","last_became_active":"2024-04-25T00:55:38.637419+0000","last_became_peered":"2024-04-25T00:55:38.637419+0000","last_unstale":"2024-04-25T00:55:40.638712+0000","last_undegraded":"2024-04-25T00:55:40.638712+0000","last_fullsized":"2024-04-25T00:55:40.638712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636532+0000","last_change":"2024-04-25T00:55:39.635022+0000","last_active":"2024-04-25T00:55:40.636532+0000","last_peered":"2024-04-25T00:55:40.636532+0000","last_clean":"2024-04-25T00:55:40.636532+0000","last_became_active":"2024-04-25T00:55:39.634107+0000","last_became_peered":"2024-04-25T00:55:39.634107+0000","last_unstale":"2024-04-25T00:55:40.636532+0000","last_undegraded":"2024-04-25T00:55:40.636532+0000","last_fullsized":"2024-04-25T00:55:40.636532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.087648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637039+0000","last_change":"2024-04-25T00:55:38.607260+0000","last_active":"2024-04-25T00:55:40.637039+0000","last_peered":"2024-04-25T00:55:40.637039+0000","last_clean":"2024-04-25T00:55:40.637039+0000","last_became_active":"2024-04-25T00:55:38.606910+0000","last_became_peered":"2024-04-25T00:55:38.606910+0000","last_unstale":"2024-04-25T00:55:40.637039+0000","last_undegraded":"2024-04-25T00:55:40.637039+0000","last_fullsized":"2024-04-25T00:55:40.637039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:25.842602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637823+0000","last_change":"2024-04-25T00:55:39.641228+0000","last_active":"2024-04-25T00:55:40.637823+0000","last_peered":"2024-04-25T00:55:40.637823+0000","last_clean":"2024-04-25T00:55:40.637823+0000","last_became_active":"2024-04-25T00:55:39.640434+0000","last_became_peered":"2024-04-25T00:55:39.640434+0000","last_unstale":"2024-04-25T00:55:40.637823+0000","last_undegraded":"2024-04-25T00:55:40.637823+0000","last_fullsized":"2024-04-25T00:55:40.637823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.552805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018046+0000","last_change":"2024-04-25T00:55:38.630316+0000","last_active":"2024-04-25T00:55:41.018046+0000","last_peered":"2024-04-25T00:55:41.018046+0000","last_clean":"2024-04-25T00:55:41.018046+0000","last_became_active":"2024-04-25T00:55:38.629417+0000","last_became_peered":"2024-04-25T00:55:38.629417+0000","last_unstale":"2024-04-25T00:55:41.018046+0000","last_undegraded":"2024-04-25T00:55:41.018046+0000","last_fullsized":"2024-04-25T00:55:41.018046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636661+0000","last_change":"2024-04-25T00:55:39.633030+0000","last_active":"2024-04-25T00:55:40.636661+0000","last_peered":"2024-04-25T00:55:40.636661+0000","last_clean":"2024-04-25T00:55:40.636661+0000","last_became_active":"2024-04-25T00:55:39.632300+0000","last_became_peered":"2024-04-25T00:55:39.632300+0000","last_unstale":"2024-04-25T00:55:40.636661+0000","last_undegraded":"2024-04-25T00:55:40.636661+0000","last_fullsized":"2024-04-25T00:55:40.636661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652457+0000","last_change":"2024-04-25T00:55:38.634050+0000","last_active":"2024-04-25T00:55:40.652457+0000","last_peered":"2024-04-25T00:55:40.652457+0000","last_clean":"2024-04-25T00:55:40.652457+0000","last_became_active":"2024-04-25T00:55:38.633672+0000","last_became_peered":"2024-04-25T00:55:38.633672+0000","last_unstale":"2024-04-25T00:55:40.652457+0000","last_undegraded":"2024-04-25T00:55:40.652457+0000","last_fullsized":"2024-04-25T00:55:40.652457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.022010+0000","last_change":"2024-04-25T00:55:39.652710+0000","last_active":"2024-04-25T00:55:41.022010+0000","last_peered":"2024-04-25T00:55:41.022010+0000","last_clean":"2024-04-25T00:55:41.022010+0000","last_became_active":"2024-04-25T00:55:39.652302+0000","last_became_peered":"2024-04-25T00:55:39.652302+0000","last_unstale":"2024-04-25T00:55:41.022010+0000","last_undegraded":"2024-04-25T00:55:41.022010+0000","last_fullsized":"2024-04-25T00:55:41.022010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652764+0000","last_change":"2024-04-25T00:55:38.636274+0000","last_active":"2024-04-25T00:55:40.652764+0000","last_peered":"2024-04-25T00:55:40.652764+0000","last_clean":"2024-04-25T00:55:40.652764+0000","last_became_active":"2024-04-25T00:55:38.635626+0000","last_became_peered":"2024-04-25T00:55:38.635626+0000","last_unstale":"2024-04-25T00:55:40.652764+0000","last_undegraded":"2024-04-25T00:55:40.652764+0000","last_fullsized":"2024-04-25T00:55:40.652764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018903+0000","last_change":"2024-04-25T00:55:39.635041+0000","last_active":"2024-04-25T00:55:41.018903+0000","last_peered":"2024-04-25T00:55:41.018903+0000","last_clean":"2024-04-25T00:55:41.018903+0000","last_became_active":"2024-04-25T00:55:39.634758+0000","last_became_peered":"2024-04-25T00:55:39.634758+0000","last_unstale":"2024-04-25T00:55:41.018903+0000","last_undegraded":"2024-04-25T00:55:41.018903+0000","last_fullsized":"2024-04-25T00:55:41.018903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:53.762656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016600+0000","last_change":"2024-04-25T00:55:38.630288+0000","last_active":"2024-04-25T00:55:41.016600+0000","last_peered":"2024-04-25T00:55:41.016600+0000","last_clean":"2024-04-25T00:55:41.016600+0000","last_became_active":"2024-04-25T00:55:38.629355+0000","last_became_peered":"2024-04-25T00:55:38.629355+0000","last_unstale":"2024-04-25T00:55:41.016600+0000","last_undegraded":"2024-04-25T00:55:41.016600+0000","last_fullsized":"2024-04-25T00:55:41.016600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636226+0000","last_change":"2024-04-25T00:55:39.623782+0000","last_active":"2024-04-25T00:55:40.636226+0000","last_peered":"2024-04-25T00:55:40.636226+0000","last_clean":"2024-04-25T00:55:40.636226+0000","last_became_active":"2024-04-25T00:55:39.622815+0000","last_became_peered":"2024-04-25T00:55:39.622815+0000","last_unstale":"2024-04-25T00:55:40.636226+0000","last_undegraded":"2024-04-25T00:55:40.636226+0000","last_fullsized":"2024-04-25T00:55:40.636226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:03.896351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637109+0000","last_change":"2024-04-25T00:55:38.618497+0000","last_active":"2024-04-25T00:55:40.637109+0000","last_peered":"2024-04-25T00:55:40.637109+0000","last_clean":"2024-04-25T00:55:40.637109+0000","last_became_active":"2024-04-25T00:55:38.618150+0000","last_became_peered":"2024-04-25T00:55:38.618150+0000","last_unstale":"2024-04-25T00:55:40.637109+0000","last_undegraded":"2024-04-25T00:55:40.637109+0000","last_fullsized":"2024-04-25T00:55:40.637109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021086+0000","last_change":"2024-04-25T00:55:39.640734+0000","last_active":"2024-04-25T00:55:41.021086+0000","last_peered":"2024-04-25T00:55:41.021086+0000","last_clean":"2024-04-25T00:55:41.021086+0000","last_became_active":"2024-04-25T00:55:39.639372+0000","last_became_peered":"2024-04-25T00:55:39.639372+0000","last_unstale":"2024-04-25T00:55:41.021086+0000","last_undegraded":"2024-04-25T00:55:41.021086+0000","last_fullsized":"2024-04-25T00:55:41.021086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636950+0000","last_change":"2024-04-25T00:55:38.613079+0000","last_active":"2024-04-25T00:55:40.636950+0000","last_peered":"2024-04-25T00:55:40.636950+0000","last_clean":"2024-04-25T00:55:40.636950+0000","last_became_active":"2024-04-25T00:55:38.612643+0000","last_became_peered":"2024-04-25T00:55:38.612643+0000","last_unstale":"2024-04-25T00:55:40.636950+0000","last_undegraded":"2024-04-25T00:55:40.636950+0000","last_fullsized":"2024-04-25T00:55:40.636950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021795+0000","last_change":"2024-04-25T00:55:39.640875+0000","last_active":"2024-04-25T00:55:41.021795+0000","last_peered":"2024-04-25T00:55:41.021795+0000","last_clean":"2024-04-25T00:55:41.021795+0000","last_became_active":"2024-04-25T00:55:39.638658+0000","last_became_peered":"2024-04-25T00:55:39.638658+0000","last_unstale":"2024-04-25T00:55:41.021795+0000","last_undegraded":"2024-04-25T00:55:41.021795+0000","last_fullsized":"2024-04-25T00:55:41.021795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017957+0000","last_change":"2024-04-25T00:55:38.624428+0000","last_active":"2024-04-25T00:55:41.017957+0000","last_peered":"2024-04-25T00:55:41.017957+0000","last_clean":"2024-04-25T00:55:41.017957+0000","last_became_active":"2024-04-25T00:55:38.622736+0000","last_became_peered":"2024-04-25T00:55:38.622736+0000","last_unstale":"2024-04-25T00:55:41.017957+0000","last_undegraded":"2024-04-25T00:55:41.017957+0000","last_fullsized":"2024-04-25T00:55:41.017957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:26.077118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652902+0000","last_change":"2024-04-25T00:55:39.631349+0000","last_active":"2024-04-25T00:55:40.652902+0000","last_peered":"2024-04-25T00:55:40.652902+0000","last_clean":"2024-04-25T00:55:40.652902+0000","last_became_active":"2024-04-25T00:55:39.630507+0000","last_became_peered":"2024-04-25T00:55:39.630507+0000","last_unstale":"2024-04-25T00:55:40.652902+0000","last_undegraded":"2024-04-25T00:55:40.652902+0000","last_fullsized":"2024-04-25T00:55:40.652902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637806+0000","last_change":"2024-04-25T00:55:39.628089+0000","last_active":"2024-04-25T00:55:40.637806+0000","last_peered":"2024-04-25T00:55:40.637806+0000","last_clean":"2024-04-25T00:55:40.637806+0000","last_became_active":"2024-04-25T00:55:39.627070+0000","last_became_peered":"2024-04-25T00:55:39.627070+0000","last_unstale":"2024-04-25T00:55:40.637806+0000","last_undegraded":"2024-04-25T00:55:40.637806+0000","last_fullsized":"2024-04-25T00:55:40.637806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017214+0000","last_change":"2024-04-25T00:55:38.634723+0000","last_active":"2024-04-25T00:55:41.017214+0000","last_peered":"2024-04-25T00:55:41.017214+0000","last_clean":"2024-04-25T00:55:41.017214+0000","last_became_active":"2024-04-25T00:55:38.633867+0000","last_became_peered":"2024-04-25T00:55:38.633867+0000","last_unstale":"2024-04-25T00:55:41.017214+0000","last_undegraded":"2024-04-25T00:55:41.017214+0000","last_fullsized":"2024-04-25T00:55:41.017214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638050+0000","last_change":"2024-04-25T00:55:39.618027+0000","last_active":"2024-04-25T00:55:40.638050+0000","last_peered":"2024-04-25T00:55:40.638050+0000","last_clean":"2024-04-25T00:55:40.638050+0000","last_became_active":"2024-04-25T00:55:39.617211+0000","last_became_peered":"2024-04-25T00:55:39.617211+0000","last_unstale":"2024-04-25T00:55:40.638050+0000","last_undegraded":"2024-04-25T00:55:40.638050+0000","last_fullsized":"2024-04-25T00:55:40.638050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:11.034367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017702+0000","last_change":"2024-04-25T00:55:38.636516+0000","last_active":"2024-04-25T00:55:41.017702+0000","last_peered":"2024-04-25T00:55:41.017702+0000","last_clean":"2024-04-25T00:55:41.017702+0000","last_became_active":"2024-04-25T00:55:38.635662+0000","last_became_peered":"2024-04-25T00:55:38.635662+0000","last_unstale":"2024-04-25T00:55:41.017702+0000","last_undegraded":"2024-04-25T00:55:41.017702+0000","last_fullsized":"2024-04-25T00:55:41.017702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018504+0000","last_change":"2024-04-25T00:55:39.622594+0000","last_active":"2024-04-25T00:55:41.018504+0000","last_peered":"2024-04-25T00:55:41.018504+0000","last_clean":"2024-04-25T00:55:41.018504+0000","last_became_active":"2024-04-25T00:55:39.621768+0000","last_became_peered":"2024-04-25T00:55:39.621768+0000","last_unstale":"2024-04-25T00:55:41.018504+0000","last_undegraded":"2024-04-25T00:55:41.018504+0000","last_fullsized":"2024-04-25T00:55:41.018504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:27.074899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021808+0000","last_change":"2024-04-25T00:55:38.609758+0000","last_active":"2024-04-25T00:55:41.021808+0000","last_peered":"2024-04-25T00:55:41.021808+0000","last_clean":"2024-04-25T00:55:41.021808+0000","last_became_active":"2024-04-25T00:55:38.609401+0000","last_became_peered":"2024-04-25T00:55:38.609401+0000","last_unstale":"2024-04-25T00:55:41.021808+0000","last_undegraded":"2024-04-25T00:55:41.021808+0000","last_fullsized":"2024-04-25T00:55:41.021808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:01.720059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021639+0000","last_change":"2024-04-25T00:55:39.657195+0000","last_active":"2024-04-25T00:55:41.021639+0000","last_peered":"2024-04-25T00:55:41.021639+0000","last_clean":"2024-04-25T00:55:41.021639+0000","last_became_active":"2024-04-25T00:55:39.656921+0000","last_became_peered":"2024-04-25T00:55:39.656921+0000","last_unstale":"2024-04-25T00:55:41.021639+0000","last_undegraded":"2024-04-25T00:55:41.021639+0000","last_fullsized":"2024-04-25T00:55:41.021639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021304+0000","last_change":"2024-04-25T00:55:38.629445+0000","last_active":"2024-04-25T00:55:41.021304+0000","last_peered":"2024-04-25T00:55:41.021304+0000","last_clean":"2024-04-25T00:55:41.021304+0000","last_became_active":"2024-04-25T00:55:38.628997+0000","last_became_peered":"2024-04-25T00:55:38.628997+0000","last_unstale":"2024-04-25T00:55:41.021304+0000","last_undegraded":"2024-04-25T00:55:41.021304+0000","last_fullsized":"2024-04-25T00:55:41.021304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:01.720059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:55:44.006 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:55:44.007 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:55:44.007 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:55:44.007 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T00:55:44.007 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T00:55:44.056 INFO:tasks.ceph.mds.b.smithi187.stderr:2024-04-25T00:55:44.052+0000 7f48af7b6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:55:44.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.152+0000 7fcc2793c640 1 Processor -- start 2024-04-25T00:55:44.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.152+0000 7fcc2793c640 1 -- start start 2024-04-25T00:55:44.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc2793c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fcc2006c630 0x7fcc200aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc2793c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc200b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc2793c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc200af760 con 0x7fcc2006ca10 2024-04-25T00:55:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc2793c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fcc20148860 con 0x7fcc2006c630 2024-04-25T00:55:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc2793c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc200b52f0 con 0x7fcc200add70 2024-04-25T00:55:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc25ed9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc200b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc25ed9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc200b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51194/0 (socket says 172.21.15.112:51194) 2024-04-25T00:55:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc25ed9640 1 -- 172.21.15.112:0/1069168647 learned_addr learned my addr 172.21.15.112:0/1069168647 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:44.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc256d8640 1 --2- 172.21.15.112:0/1069168647 >> v2:172.21.15.187:3300/0 conn(0x7fcc2006c630 0x7fcc200aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc25ed9640 1 -- 172.21.15.112:0/1069168647 >> v2:172.21.15.187:3300/0 conn(0x7fcc2006c630 msgr2=0x7fcc200aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:44.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc25ed9640 1 --2- 172.21.15.112:0/1069168647 >> v2:172.21.15.187:3300/0 conn(0x7fcc2006c630 0x7fcc200aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc25ed9640 1 -- 172.21.15.112:0/1069168647 >> v1:172.21.15.187:6789/0 conn(0x7fcc2006ca10 legacy=0x7fcc201477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc25ed9640 1 -- 172.21.15.112:0/1069168647 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc200afa00 con 0x7fcc200add70 2024-04-25T00:55:44.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.156+0000 7fcc25ed9640 1 --2- 172.21.15.112:0/1069168647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc200b4db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009e50 tx=0x7fcc1801c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2711c83e649c88f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc17fff640 1 -- 172.21.15.112:0/1069168647 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc180a6500 con 0x7fcc200add70 2024-04-25T00:55:44.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc17fff640 1 -- 172.21.15.112:0/1069168647 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc180a66a0 con 0x7fcc200add70 2024-04-25T00:55:44.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc17fff640 1 -- 172.21.15.112:0/1069168647 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc180a6a70 con 0x7fcc200add70 2024-04-25T00:55:44.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- 172.21.15.112:0/1069168647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 msgr2=0x7fcc200b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 --2- 172.21.15.112:0/1069168647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc200b4db0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009e50 tx=0x7fcc1801c910 comp rx=0 tx=0).stop 2024-04-25T00:55:44.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- 172.21.15.112:0/1069168647 shutdown_connections 2024-04-25T00:55:44.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 --2- 172.21.15.112:0/1069168647 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc200b4db0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 --2- 172.21.15.112:0/1069168647 >> v2:172.21.15.187:3300/0 conn(0x7fcc2006c630 0x7fcc200aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- 172.21.15.112:0/1069168647 >> 172.21.15.112:0/1069168647 conn(0x7fcc200b5980 msgr2=0x7fcc200b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:44.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- 172.21.15.112:0/1069168647 shutdown_connections 2024-04-25T00:55:44.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- 172.21.15.112:0/1069168647 wait complete. 2024-04-25T00:55:44.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 Processor -- start 2024-04-25T00:55:44.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- start start 2024-04-25T00:55:44.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc2006c630 0x7fcc201e0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc2006ca10 0x7fcc2013c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc2013ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2014f9e0 con 0x7fcc2006ca10 2024-04-25T00:55:44.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2014f860 con 0x7fcc2006c630 2024-04-25T00:55:44.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc2014fb60 con 0x7fcc200add70 2024-04-25T00:55:44.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc2006ca10 0x7fcc2013c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc2006ca10 0x7fcc2013c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51026/0 (socket says 172.21.15.112:51026) 2024-04-25T00:55:44.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 -- 172.21.15.112:0/3988155266 learned_addr learned my addr 172.21.15.112:0/3988155266 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:44.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc25ed9640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc2013ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc256d8640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc2006c630 0x7fcc201e0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 -- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 msgr2=0x7fcc2013ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc2013ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 -- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc2006c630 msgr2=0x7fcc201e0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc2006c630 0x7fcc201e0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 -- 172.21.15.112:0/3988155266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2013b8a0 con 0x7fcc2006ca10 2024-04-25T00:55:44.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc256d8640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc2006c630 0x7fcc201e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:44.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc24ed7640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc2006ca10 0x7fcc2013c8f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000c420 tx=0x7fcc1000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc15ffb640 1 -- 172.21.15.112:0/3988155266 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10017070 con 0x7fcc2006ca10 2024-04-25T00:55:44.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc15ffb640 1 -- 172.21.15.112:0/3988155266 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc100052d0 con 0x7fcc2006ca10 2024-04-25T00:55:44.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc201e1320 con 0x7fcc2006ca10 2024-04-25T00:55:44.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc15ffb640 1 -- 172.21.15.112:0/3988155266 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc100054c0 con 0x7fcc2006ca10 2024-04-25T00:55:44.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.160+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc201515f0 con 0x7fcc2006ca10 2024-04-25T00:55:44.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.164+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2004f580 con 0x7fcc2006ca10 2024-04-25T00:55:44.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.164+0000 7fcc15ffb640 1 -- 172.21.15.112:0/3988155266 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcc1001a070 con 0x7fcc2006ca10 2024-04-25T00:55:44.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.164+0000 7fcc15ffb640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc040678c0 0x7fcc04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.164+0000 7fcc256d8640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc040678c0 0x7fcc04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.164+0000 7fcc15ffb640 1 -- 172.21.15.112:0/3988155266 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc10084fa0 con 0x7fcc2006ca10 2024-04-25T00:55:44.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.164+0000 7fcc256d8640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc040678c0 0x7fcc04069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009ae0 tx=0x7fcc1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.168+0000 7fcc15ffb640 1 -- 172.21.15.112:0/3988155266 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcc10055b70 con 0x7fcc2006ca10 2024-04-25T00:55:44.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.348+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fcc20141b20 con 0x7fcc2006ca10 2024-04-25T00:55:44.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.356+0000 7fcc15ffb640 1 -- 172.21.15.112:0/3988155266 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166888 (secure 0 0 0) 0x7fcc10055470 con 0x7fcc2006ca10 2024-04-25T00:55:44.363 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:44.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc040678c0 msgr2=0x7fcc04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc040678c0 0x7fcc04069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009ae0 tx=0x7fcc1c008040 comp rx=0 tx=0).stop 2024-04-25T00:55:44.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc2006ca10 msgr2=0x7fcc2013c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc2006ca10 0x7fcc2013c8f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000c420 tx=0x7fcc1000c8e0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 shutdown_connections 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc200add70 0x7fcc2013ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc2006ca10 0x7fcc2013c8f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc040678c0 0x7fcc04069d80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 --2- 172.21.15.112:0/3988155266 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc2006c630 0x7fcc201e0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 >> 172.21.15.112:0/3988155266 conn(0x7fcc200b5980 msgr2=0x7fcc200b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 shutdown_connections 2024-04-25T00:55:44.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.360+0000 7fcc2793c640 1 -- 172.21.15.112:0/3988155266 wait complete. 2024-04-25T00:55:44.380 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":80465356},{"type":"v1","addr":"172.21.15.112:6833","nonce":80465356}]},"active_addr":"172.21.15.112:6833/80465356","active_change":"2024-04-25T00:55:19.501326+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3430938356}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1617407078}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1225316158}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3044331864}]}]} 2024-04-25T00:55:44.383 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T00:55:44.383 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T00:55:44.383 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:44.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.492+0000 7f3b2d52d640 1 Processor -- start 2024-04-25T00:55:44.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.492+0000 7f3b2d52d640 1 -- start start 2024-04-25T00:55:44.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b2d52d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 0x7f3b280aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b2d52d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b281485e0 con 0x7f3b2806ca10 2024-04-25T00:55:44.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b2d52d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b280af760 con 0x7f3b2806c630 2024-04-25T00:55:44.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b2d52d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b28148860 con 0x7f3b280add70 2024-04-25T00:55:44.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b267fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3b2806ca10 0x7f3b28147ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57230/0 (socket says 172.21.15.112:57230) 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b267fc640 1 -- 172.21.15.112:0/3997560403 learned_addr learned my addr 172.21.15.112:0/3997560403 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b277fe640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 0x7f3b280aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78523101 0 0) 0x7f3b281485e0 con 0x7f3b2806ca10 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b10003530 con 0x7f3b2806ca10 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 370845491 0 0) 0x7f3b10003530 con 0x7f3b2806ca10 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 msgr2=0x7f3b280aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 0x7f3b280aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 >> v1:172.21.15.187:6789/0 conn(0x7f3b2806c630 legacy=0x7f3b281477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b280afa00 con 0x7f3b2806ca10 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f3b1c001680 con 0x7f3b2806ca10 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 >> v1:172.21.15.112:6789/0 conn(0x7f3b2806ca10 legacy=0x7f3b28147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b100039f0 0x7f3b10003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10004810 0x7f3b10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 0x7f3b100052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1000bb60 con 0x7f3b100039f0 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1000bd10 con 0x7f3b10004810 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b26ffd640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b100039f0 0x7f3b10003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b1000bf10 con 0x7f3b280add70 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b267fc640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10004810 0x7f3b10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b277fe640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 0x7f3b100052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b26ffd640 1 -- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 msgr2=0x7f3b100052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b26ffd640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 0x7f3b100052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b26ffd640 1 -- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10004810 msgr2=0x7f3b10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b26ffd640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10004810 0x7f3b10004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b26ffd640 1 -- 172.21.15.112:0/3997560403 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b10003530 con 0x7f3b100039f0 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b267fc640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b10004810 0x7f3b10004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:44.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b277fe640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b280add70 0x7f3b100052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:44.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b26ffd640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b100039f0 0x7f3b10003e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001780 tx=0x7f3b1801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231bbfe6d9391370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b180013a0 con 0x7f3b100039f0 2024-04-25T00:55:44.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b180052f0 con 0x7f3b100039f0 2024-04-25T00:55:44.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.496+0000 7f3b25ffb640 1 -- 172.21.15.112:0/3997560403 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18003840 con 0x7f3b100039f0 2024-04-25T00:55:44.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- 172.21.15.112:0/3997560403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b100039f0 msgr2=0x7f3b10003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 --2- 172.21.15.112:0/3997560403 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b100039f0 0x7f3b10003e10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001780 tx=0x7f3b1801c420 comp rx=0 tx=0).stop 2024-04-25T00:55:44.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 -- 172.21.15.112:0/3997560403 reap_dead start 2024-04-25T00:55:44.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- 172.21.15.112:0/3997560403 shutdown_connections 2024-04-25T00:55:44.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- 172.21.15.112:0/3997560403 >> 172.21.15.112:0/3997560403 conn(0x7f3b280b5980 msgr2=0x7f3b280b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:44.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- 172.21.15.112:0/3997560403 shutdown_connections 2024-04-25T00:55:44.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- 172.21.15.112:0/3997560403 wait complete. 2024-04-25T00:55:44.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 Processor -- start 2024-04-25T00:55:44.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- start start 2024-04-25T00:55:44.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b100039f0 0x7f3b28071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b280707c0 0x7f3b28070c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2806f6c0 0x7f3b2806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b280b4060 con 0x7f3b2806f6c0 2024-04-25T00:55:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b280b3ee0 con 0x7f3b280707c0 2024-04-25T00:55:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b280b41e0 con 0x7f3b100039f0 2024-04-25T00:55:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2806f6c0 0x7f3b2806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2806f6c0 0x7f3b2806fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51042/0 (socket says 172.21.15.112:51042) 2024-04-25T00:55:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 -- 172.21.15.112:0/602370429 learned_addr learned my addr 172.21.15.112:0/602370429 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:44.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b267fc640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b280707c0 0x7f3b28070c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b26ffd640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b100039f0 0x7f3b28071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 -- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b100039f0 msgr2=0x7f3b28071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b100039f0 0x7f3b28071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 -- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b280707c0 msgr2=0x7f3b28070c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b280707c0 0x7f3b28070c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 -- 172.21.15.112:0/602370429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b28143af0 con 0x7f3b2806f6c0 2024-04-25T00:55:44.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b26ffd640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b100039f0 0x7f3b28071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b277fe640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2806f6c0 0x7f3b2806fb40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b280afe50 tx=0x7f3b2000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b0f7fe640 1 -- 172.21.15.112:0/602370429 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b2001c070 con 0x7f3b2806f6c0 2024-04-25T00:55:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b0f7fe640 1 -- 172.21.15.112:0/602370429 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b2000fe90 con 0x7f3b2806f6c0 2024-04-25T00:55:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b0f7fe640 1 -- 172.21.15.112:0/602370429 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b20017620 con 0x7f3b2806f6c0 2024-04-25T00:55:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b28144ea0 con 0x7f3b2806f6c0 2024-04-25T00:55:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.500+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2814d8b0 con 0x7f3b2806f6c0 2024-04-25T00:55:44.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.504+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2804f580 con 0x7f3b2806f6c0 2024-04-25T00:55:44.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.504+0000 7f3b0f7fe640 1 -- 172.21.15.112:0/602370429 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3b2001f070 con 0x7f3b2806f6c0 2024-04-25T00:55:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.504+0000 7f3b0f7fe640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b140678c0 0x7f3b14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.504+0000 7f3b26ffd640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b140678c0 0x7f3b14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.504+0000 7f3b0f7fe640 1 -- 172.21.15.112:0/602370429 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3b20085d80 con 0x7f3b2806f6c0 2024-04-25T00:55:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.504+0000 7f3b26ffd640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b140678c0 0x7f3b14069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001750 tx=0x7f3b180011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.508+0000 7f3b0f7fe640 1 -- 172.21.15.112:0/602370429 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3b20012070 con 0x7f3b2806f6c0 2024-04-25T00:55:44.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.656+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b28141b20 con 0x7f3b2806f6c0 2024-04-25T00:55:44.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.660+0000 7f3b0f7fe640 1 -- 172.21.15.112:0/602370429 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f3b20056950 con 0x7f3b2806f6c0 2024-04-25T00:55:44.665 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:44.665 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:44.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b140678c0 msgr2=0x7f3b14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b140678c0 0x7f3b14069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3b18001750 tx=0x7f3b180011a0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2806f6c0 msgr2=0x7f3b2806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2806f6c0 0x7f3b2806fb40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b280afe50 tx=0x7f3b2000f4a0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 shutdown_connections 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b2806f6c0 0x7f3b2806fb40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b140678c0 0x7f3b14069d80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b280707c0 0x7f3b28070c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 --2- 172.21.15.112:0/602370429 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b100039f0 0x7f3b28071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 >> 172.21.15.112:0/602370429 conn(0x7f3b280b5980 msgr2=0x7f3b281509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 shutdown_connections 2024-04-25T00:55:44.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.664+0000 7f3b2d52d640 1 -- 172.21.15.112:0/602370429 wait complete. 2024-04-25T00:55:44.683 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T00:55:44.683 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.792+0000 7f693294b640 1 Processor -- start 2024-04-25T00:55:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.792+0000 7f693294b640 1 -- start start 2024-04-25T00:55:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.792+0000 7f693294b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c0add70 0x7f692c145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.792+0000 7f693294b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f692c0b3ce0 0x7f692c14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.792+0000 7f693294b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f692c0aeec0 con 0x7f692c0b23f0 2024-04-25T00:55:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.792+0000 7f693294b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f692c0af780 con 0x7f692c0b3ce0 2024-04-25T00:55:44.807 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.792+0000 7f693294b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f692c0b4170 con 0x7f692c0add70 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f692c0b23f0 0x7f692c072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57246/0 (socket says 172.21.15.112:57246) 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 -- 172.21.15.112:0/4285104395 learned_addr learned my addr 172.21.15.112:0/4285104395 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692b7fe640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c0add70 0x7f692c145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1511434484 0 0) 0x7f692c0aeec0 con 0x7f692c0b23f0 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f6930ee8640 1 --2- 172.21.15.112:0/4285104395 >> v2:172.21.15.187:3300/0 conn(0x7f692c0b3ce0 0x7f692c14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6914003530 con 0x7f692c0b23f0 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3756316514 0 0) 0x7f6914003530 con 0x7f692c0b23f0 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c0add70 msgr2=0x7f692c145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c0add70 0x7f692c145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 >> v2:172.21.15.187:3300/0 conn(0x7f692c0b3ce0 msgr2=0x7f692c14fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 --2- 172.21.15.112:0/4285104395 >> v2:172.21.15.187:3300/0 conn(0x7f692c0b3ce0 0x7f692c14fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c0b27d0 con 0x7f692c0b23f0 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f6920001680 con 0x7f692c0b23f0 2024-04-25T00:55:44.808 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f6930ee8640 1 --2- 172.21.15.112:0/4285104395 >> v2:172.21.15.187:3300/0 conn(0x7f692c0b3ce0 0x7f692c14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 >> v1:172.21.15.112:6789/0 conn(0x7f692c0b23f0 legacy=0x7f692c072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140039c0 0x7f6914003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6914004810 0x7f6914004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c0add70 0x7f69140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0af780 con 0x7f69140039c0 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0b4170 con 0x7f6914004810 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f691400ba00 con 0x7f692c0add70 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140039c0 0x7f6914003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692b7fe640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6914004810 0x7f6914004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f6930ee8640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c0add70 0x7f69140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 -- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c0add70 msgr2=0x7f69140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c0add70 0x7f69140051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 -- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6914004810 msgr2=0x7f6914004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:44.809 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6914004810 0x7f6914004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 -- 172.21.15.112:0/4285104395 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6914003530 con 0x7f69140039c0 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692bfff640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140039c0 0x7f6914003de0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f692000c210 tx=0x7f692001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbf2a2aabda07d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920007d60 con 0x7f69140039c0 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69200a9020 con 0x7f69140039c0 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f692affd640 1 -- 172.21.15.112:0/4285104395 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920005b20 con 0x7f69140039c0 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f693294b640 1 -- 172.21.15.112:0/4285104395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140039c0 msgr2=0x7f6914003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f693294b640 1 --2- 172.21.15.112:0/4285104395 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f69140039c0 0x7f6914003de0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f692000c210 tx=0x7f692001cca0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f6930ee8640 1 -- 172.21.15.112:0/4285104395 reap_dead start 2024-04-25T00:55:44.810 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f693294b640 1 -- 172.21.15.112:0/4285104395 shutdown_connections 2024-04-25T00:55:44.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f693294b640 1 -- 172.21.15.112:0/4285104395 >> 172.21.15.112:0/4285104395 conn(0x7f692c0b5980 msgr2=0x7f692c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:44.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f693294b640 1 -- 172.21.15.112:0/4285104395 shutdown_connections 2024-04-25T00:55:44.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.796+0000 7f693294b640 1 -- 172.21.15.112:0/4285104395 wait complete. 2024-04-25T00:55:44.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 Processor -- start 2024-04-25T00:55:44.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 -- start start 2024-04-25T00:55:44.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69140039c0 0x7f692c1e1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692c071df0 0x7f692c148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.811 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c13e8e0 0x7f692c13ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0b5150 con 0x7f692c071df0 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f692c0ae150 con 0x7f69140039c0 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f692c0b52d0 con 0x7f692c13e8e0 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692c071df0 0x7f692c148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692c071df0 0x7f692c148ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51048/0 (socket says 172.21.15.112:51048) 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 -- 172.21.15.112:0/1624597071 learned_addr learned my addr 172.21.15.112:0/1624597071 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f6930ee8640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c13e8e0 0x7f692c13ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 -- 172.21.15.112:0/1624597071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c13e8e0 msgr2=0x7f692c13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c13e8e0 0x7f692c13ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 -- 172.21.15.112:0/1624597071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69140039c0 msgr2=0x7f692c1e1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69140039c0 0x7f692c1e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 -- 172.21.15.112:0/1624597071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c1e1150 con 0x7f692c071df0 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f692b7fe640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692c071df0 0x7f692c148ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f691c011610 tx=0x7f691c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.812 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f6928ff9640 1 -- 172.21.15.112:0/1624597071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c00f8f0 con 0x7f692c071df0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f6928ff9640 1 -- 172.21.15.112:0/1624597071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c00fa90 con 0x7f692c071df0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692c141120 con 0x7f692c071df0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f6928ff9640 1 -- 172.21.15.112:0/1624597071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c00fe40 con 0x7f692c071df0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692c072760 con 0x7f692c071df0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692c04f580 con 0x7f692c071df0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.800+0000 7f6928ff9640 1 -- 172.21.15.112:0/1624597071 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f691c015460 con 0x7f692c071df0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.804+0000 7f6928ff9640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69180678f0 0x7f6918069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.804+0000 7f692bfff640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69180678f0 0x7f6918069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.804+0000 7f6928ff9640 1 -- 172.21.15.112:0/1624597071 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f691c0865b0 con 0x7f692c071df0 2024-04-25T00:55:44.813 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.804+0000 7f692bfff640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69180678f0 0x7f6918069db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69200aa000 tx=0x7f692c0ae380 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:44.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.808+0000 7f6928ff9640 1 -- 172.21.15.112:0/1624597071 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f691c019030 con 0x7f692c071df0 2024-04-25T00:55:44.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.952+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f692c073700 con 0x7f692c071df0 2024-04-25T00:55:44.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.956+0000 7f6928ff9640 1 -- 172.21.15.112:0/1624597071 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f691c057180 con 0x7f692c071df0 2024-04-25T00:55:44.961 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:44.961 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:44.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69180678f0 msgr2=0x7f6918069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69180678f0 0x7f6918069db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69200aa000 tx=0x7f692c0ae380 comp rx=0 tx=0).stop 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692c071df0 msgr2=0x7f692c148ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692c071df0 0x7f692c148ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f691c011610 tx=0x7f691c011ad0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 shutdown_connections 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692c13e8e0 0x7f692c13ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692c071df0 0x7f692c148ec0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69180678f0 0x7f6918069db0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 --2- 172.21.15.112:0/1624597071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69140039c0 0x7f692c1e1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 >> 172.21.15.112:0/1624597071 conn(0x7f692c0b5980 msgr2=0x7f692c0b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 shutdown_connections 2024-04-25T00:55:44.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:44.960+0000 7f693294b640 1 -- 172.21.15.112:0/1624597071 wait complete. 2024-04-25T00:55:44.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T00:55:44.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T00:55:44.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T00:55:44.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T00:55:44.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T00:55:44.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T00:55:44.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T00:55:44.982 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T00:55:45.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc256b1c640 1 Processor -- start 2024-04-25T00:55:45.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10cd76a640 1 Processor -- start 2024-04-25T00:55:45.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10cd76a640 1 -- start start 2024-04-25T00:55:45.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc256b1c640 1 -- start start 2024-04-25T00:55:45.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc256b1c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc2500b4a70 0x7fc2500b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc256b1c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc2500af0e0 0x7fc250146d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc256b1c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2500b4e50 con 0x7fc250147260 2024-04-25T00:55:45.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc256b1c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc2500aec50 con 0x7fc2500af0e0 2024-04-25T00:55:45.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc256b1c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc2500b0400 con 0x7fc2500b4a70 2024-04-25T00:55:45.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc24ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc2500af0e0 0x7fc250146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc24ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc2500af0e0 0x7fc250146d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51054/0 (socket says 172.21.15.112:51054) 2024-04-25T00:55:45.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc24ffff640 1 -- 172.21.15.112:0/2337614767 learned_addr learned my addr 172.21.15.112:0/2337614767 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc2548b8640 1 --2- 172.21.15.112:0/2337614767 >> v2:172.21.15.187:3300/0 conn(0x7fc2500b4a70 0x7fc2500b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc24ffff640 1 -- 172.21.15.112:0/2337614767 >> v2:172.21.15.187:3300/0 conn(0x7fc2500b4a70 msgr2=0x7fc2500b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc24ffff640 1 --2- 172.21.15.112:0/2337614767 >> v2:172.21.15.187:3300/0 conn(0x7fc2500b4a70 0x7fc2500b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc24ffff640 1 -- 172.21.15.112:0/2337614767 >> v1:172.21.15.187:6789/0 conn(0x7fc250147260 legacy=0x7fc25006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7fc24ffff640 1 -- 172.21.15.112:0/2337614767 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2500b06e0 con 0x7fc2500af0e0 2024-04-25T00:55:45.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10cd76a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10c80b48c0 0x7f10c80add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10cd76a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f10c8072760 0x7f10c8072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10cd76a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10c8145320 0x7f10c8145720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10cd76a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f10c806e010 con 0x7f10c80b48c0 2024-04-25T00:55:45.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10cd76a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f10c80af5c0 con 0x7f10c8072760 2024-04-25T00:55:45.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10cd76a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10c813faa0 con 0x7f10c8145320 2024-04-25T00:55:45.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f59c3fff640 1 Processor -- start 2024-04-25T00:55:45.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f59c3fff640 1 -- start start 2024-04-25T00:55:45.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10c6ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10c80b48c0 0x7f10c80add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10c6ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10c80b48c0 0x7f10c80add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51066/0 (socket says 172.21.15.112:51066) 2024-04-25T00:55:45.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10c6ffd640 1 -- 172.21.15.112:0/3920809561 learned_addr learned my addr 172.21.15.112:0/3920809561 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f10c67fc640 1 --2- 172.21.15.112:0/3920809561 >> v2:172.21.15.187:3300/0 conn(0x7f10c8072760 0x7f10c8072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f3175296640 1 Processor -- start 2024-04-25T00:55:45.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f9e9f58e640 1 Processor -- start 2024-04-25T00:55:45.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f3175296640 1 -- start start 2024-04-25T00:55:45.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f10c77fe640 1 --2- 172.21.15.112:0/3920809561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10c8145320 0x7f10c8145720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a13919640 1 Processor -- start 2024-04-25T00:55:45.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a13919640 1 -- start start 2024-04-25T00:55:45.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 0x7f59c41432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f59c41472a0 0x7f59c40afe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f59c40b57d0 0x7f59c40b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.152+0000 7f9e9f58e640 1 -- start start 2024-04-25T00:55:45.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9f58e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e980b73e0 0x7f9e98145120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9f58e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9e980b6610 0x7f9e9814f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9f58e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e980b5480 con 0x7f9e980b5d90 2024-04-25T00:55:45.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9f58e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9e9813f8e0 con 0x7f9e980b73e0 2024-04-25T00:55:45.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9f58e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9e980b5760 con 0x7f9e980b6610 2024-04-25T00:55:45.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c3fff640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f59c413daa0 con 0x7f59c41472a0 2024-04-25T00:55:45.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c3fff640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f59c4146850 con 0x7f59c40b57d0 2024-04-25T00:55:45.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c3fff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59c4146e10 con 0x7f59c40728a0 2024-04-25T00:55:45.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c2ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 0x7f59c41432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c27fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f59c41472a0 0x7f59c40afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f3175296640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3168036aa0 0x7f31680d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f3175296640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3168036e80 con 0x7f31680cab10 2024-04-25T00:55:45.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f3175296640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3168029bb0 con 0x7f31680d10b0 2024-04-25T00:55:45.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f3175296640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3168038390 con 0x7f3168036aa0 2024-04-25T00:55:45.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c2ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 0x7f59c41432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51270/0 (socket says 172.21.15.112:51270) 2024-04-25T00:55:45.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c27fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f59c41472a0 0x7f59c40afe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51068/0 (socket says 172.21.15.112:51068) 2024-04-25T00:55:45.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c2ffd640 1 -- 172.21.15.112:0/2209043134 learned_addr learned my addr 172.21.15.112:0/2209043134 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316f7fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f31680d10b0 0x7f31680ca400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:52696/0 (socket says 172.21.15.112:52696) 2024-04-25T00:55:45.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316f7fe640 1 -- 172.21.15.112:0/329810591 learned_addr learned my addr 172.21.15.112:0/329810591 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:45.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c37fe640 1 --2- 172.21.15.112:0/2209043134 >> v2:172.21.15.187:3300/0 conn(0x7f59c40b57d0 0x7f59c40b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316ffff640 1 --2- 172.21.15.112:0/329810591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3168036aa0 0x7f31680d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9db2b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9e980b6610 0x7f9e9814f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9db2b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9e980b6610 0x7f9e9814f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40830/0 (socket says 172.21.15.112:40830) 2024-04-25T00:55:45.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9db2b640 1 -- 172.21.15.112:0/4226184557 learned_addr learned my addr 172.21.15.112:0/4226184557 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a13919640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1a0c0b47c0 0x7f1a0c156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a13919640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c0b2e00 con 0x7f1a0c14c3a0 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a13919640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0c0afad0 con 0x7f1a0c14bfc0 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a13919640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1a0c0b04f0 con 0x7f1a0c0b47c0 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c27fc640 1 -- 172.21.15.112:0/2209043134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 msgr2=0x7f59c41432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c27fc640 1 --2- 172.21.15.112:0/2209043134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 0x7f59c41432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c27fc640 1 -- 172.21.15.112:0/2209043134 >> v2:172.21.15.187:3300/0 conn(0x7f59c40b57d0 msgr2=0x7f59c40b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c27fc640 1 --2- 172.21.15.112:0/2209043134 >> v2:172.21.15.187:3300/0 conn(0x7f59c40b57d0 0x7f59c40b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f59c27fc640 1 -- 172.21.15.112:0/2209043134 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c4146aa0 con 0x7f59c41472a0 2024-04-25T00:55:45.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f10c6ffd640 1 -- 172.21.15.112:0/3920809561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10c8145320 msgr2=0x7f10c8145720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f10c6ffd640 1 --2- 172.21.15.112:0/3920809561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10c8145320 0x7f10c8145720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f10c6ffd640 1 -- 172.21.15.112:0/3920809561 >> v2:172.21.15.187:3300/0 conn(0x7f10c8072760 msgr2=0x7f10c8072bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f10c6ffd640 1 --2- 172.21.15.112:0/3920809561 >> v2:172.21.15.187:3300/0 conn(0x7f10c8072760 0x7f10c8072bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f10c6ffd640 1 -- 172.21.15.112:0/3920809561 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c80af7f0 con 0x7f10c80b48c0 2024-04-25T00:55:45.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f10c67fc640 1 --2- 172.21.15.112:0/3920809561 >> v2:172.21.15.187:3300/0 conn(0x7f10c8072760 0x7f10c8072bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:45.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a10eb4640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1a0c14c3a0 0x7f1a0c0b26f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57270/0 (socket says 172.21.15.112:57270) 2024-04-25T00:55:45.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a10eb4640 1 -- 172.21.15.112:0/479107762 learned_addr learned my addr 172.21.15.112:0/479107762 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:45.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a11eb6640 1 --2- 172.21.15.112:0/479107762 >> v2:172.21.15.112:3300/0 conn(0x7f1a0c0b47c0 0x7f1a0c156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316effd640 1 -- 172.21.15.112:0/329810591 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 184298444 0 0) 0x7f3168029bb0 con 0x7f31680d10b0 2024-04-25T00:55:45.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316effd640 1 -- 172.21.15.112:0/329810591 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3160003530 con 0x7f31680d10b0 2024-04-25T00:55:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316effd640 1 -- 172.21.15.112:0/329810591 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1378869544 0 0) 0x7f3168036e80 con 0x7f31680cab10 2024-04-25T00:55:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316effd640 1 -- 172.21.15.112:0/329810591 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3168029bb0 con 0x7f31680cab10 2024-04-25T00:55:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316ffff640 1 -- 172.21.15.112:0/329810591 >> v1:172.21.15.187:6789/0 conn(0x7f31680d10b0 legacy=0x7f31680ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316ffff640 1 -- 172.21.15.112:0/329810591 >> v1:172.21.15.112:6789/0 conn(0x7f31680cab10 legacy=0x7f31680db300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7fc24ffff640 1 --2- 172.21.15.112:0/2337614767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2500af0e0 0x7fc250146d20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc25006d500 tx=0x7fc24001e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f693298094ca8fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f316ffff640 1 -- 172.21.15.112:0/329810591 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31680d4ba0 con 0x7f3168036aa0 2024-04-25T00:55:45.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2098817646 0 0) 0x7f1a0c0b2e00 con 0x7f1a0c14c3a0 2024-04-25T00:55:45.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19f8003530 con 0x7f1a0c14c3a0 2024-04-25T00:55:45.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e8ffff640 1 -- 172.21.15.112:0/4226184557 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3326816888 0 0) 0x7f9e980b5480 con 0x7f9e980b5d90 2024-04-25T00:55:45.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7fc24f7fe640 1 -- 172.21.15.112:0/2337614767 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2400ad070 con 0x7fc2500af0e0 2024-04-25T00:55:45.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e8ffff640 1 -- 172.21.15.112:0/4226184557 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e84003530 con 0x7f9e980b5d90 2024-04-25T00:55:45.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9db2b640 1 -- 172.21.15.112:0/4226184557 >> v2:172.21.15.112:3300/0 conn(0x7f9e980b73e0 msgr2=0x7f9e98145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9db2b640 1 --2- 172.21.15.112:0/4226184557 >> v2:172.21.15.112:3300/0 conn(0x7f9e980b73e0 0x7f9e98145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9db2b640 1 -- 172.21.15.112:0/4226184557 >> v1:172.21.15.187:6789/0 conn(0x7f9e980b5d90 legacy=0x7f9e980b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2057557121 0 0) 0x7f1a0c0afad0 con 0x7f1a0c14bfc0 2024-04-25T00:55:45.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 --> v1:172.21.15.187:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a0c0b2e00 con 0x7f1a0c14bfc0 2024-04-25T00:55:45.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f9e9db2b640 1 -- 172.21.15.112:0/4226184557 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e980b6ac0 con 0x7f9e980b6610 2024-04-25T00:55:45.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2613574106 0 0) 0x7f19f8003530 con 0x7f1a0c14c3a0 2024-04-25T00:55:45.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 >> v2:172.21.15.112:3300/0 conn(0x7f1a0c0b47c0 msgr2=0x7f1a0c156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 --2- 172.21.15.112:0/479107762 >> v2:172.21.15.112:3300/0 conn(0x7f1a0c0b47c0 0x7f1a0c156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 >> v1:172.21.15.187:6789/0 conn(0x7f1a0c14bfc0 legacy=0x7f1a0c06c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c0b0740 con 0x7f1a0c14c3a0 2024-04-25T00:55:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f1a08001680 con 0x7f1a0c14c3a0 2024-04-25T00:55:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 >> v1:172.21.15.112:6789/0 conn(0x7f1a0c14c3a0 legacy=0x7f1a0c0b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8003ba0 0x7f19f8003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8004810 0x7f19f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19f8005220 0x7f19f800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b04f0 con 0x7f19f8003ba0 2024-04-25T00:55:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f19f800c150 con 0x7f19f8004810 2024-04-25T00:55:45.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f19f800c350 con 0x7f19f8005220 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a116b5640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8003ba0 0x7f19f8003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a10eb4640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8004810 0x7f19f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a116b5640 1 -- 172.21.15.112:0/479107762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19f8005220 msgr2=0x7f19f800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a116b5640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f19f8005220 0x7f19f800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a116b5640 1 -- 172.21.15.112:0/479107762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8004810 msgr2=0x7f19f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a116b5640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8004810 0x7f19f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a116b5640 1 -- 172.21.15.112:0/479107762 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f8003530 con 0x7f19f8003ba0 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f1a10eb4640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8004810 0x7f19f8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:45.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7fc24f7fe640 1 -- 172.21.15.112:0/2337614767 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2400a7960 con 0x7fc2500af0e0 2024-04-25T00:55:45.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7fc24f7fe640 1 -- 172.21.15.112:0/2337614767 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2400a7d30 con 0x7fc2500af0e0 2024-04-25T00:55:45.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7fc256b1c640 1 -- 172.21.15.112:0/2337614767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2500af0e0 msgr2=0x7fc250146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7fc256b1c640 1 --2- 172.21.15.112:0/2337614767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2500af0e0 0x7fc250146d20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc25006d500 tx=0x7fc24001e9a0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.156+0000 7f10c6ffd640 1 --2- 172.21.15.112:0/3920809561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 0x7f10c80add70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f10c80af9c0 tx=0x7f10b801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6b285316a038b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c27fc640 1 --2- 172.21.15.112:0/2209043134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59c41472a0 0x7f59c40afe30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f59c406ee10 tx=0x7f59b801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357733eb74073eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10c5ffb640 1 -- 172.21.15.112:0/3920809561 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b80b0070 con 0x7f10c80b48c0 2024-04-25T00:55:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10c5ffb640 1 -- 172.21.15.112:0/3920809561 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b80ab600 con 0x7f10c80b48c0 2024-04-25T00:55:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10c5ffb640 1 -- 172.21.15.112:0/3920809561 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b80ab9d0 con 0x7f10c80b48c0 2024-04-25T00:55:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 -- 172.21.15.112:0/3920809561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 msgr2=0x7f10c80add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 --2- 172.21.15.112:0/3920809561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 0x7f10c80add70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f10c80af9c0 tx=0x7f10b801c4b0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c1ffb640 1 -- 172.21.15.112:0/2209043134 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b80b0070 con 0x7f59c41472a0 2024-04-25T00:55:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c1ffb640 1 -- 172.21.15.112:0/2209043134 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b80ab600 con 0x7f59c41472a0 2024-04-25T00:55:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c1ffb640 1 -- 172.21.15.112:0/2209043134 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b80ab9d0 con 0x7f59c41472a0 2024-04-25T00:55:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 -- 172.21.15.112:0/2209043134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59c41472a0 msgr2=0x7f59c40afe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 --2- 172.21.15.112:0/2209043134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59c41472a0 0x7f59c40afe30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f59c406ee10 tx=0x7f59b801c4c0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 -- 172.21.15.112:0/2209043134 shutdown_connections 2024-04-25T00:55:45.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 --2- 172.21.15.112:0/2209043134 >> v2:172.21.15.187:3300/0 conn(0x7f59c40b57d0 0x7f59c40b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 --2- 172.21.15.112:0/2209043134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59c41472a0 0x7f59c40afe30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 --2- 172.21.15.112:0/2209043134 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 0x7f59c41432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 -- 172.21.15.112:0/2209043134 >> 172.21.15.112:0/2209043134 conn(0x7f59c406c0a0 msgr2=0x7f59c406c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 -- 172.21.15.112:0/2209043134 shutdown_connections 2024-04-25T00:55:45.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f1a11eb6640 1 -- 172.21.15.112:0/479107762 reap_dead start 2024-04-25T00:55:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 -- 172.21.15.112:0/3920809561 shutdown_connections 2024-04-25T00:55:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 --2- 172.21.15.112:0/3920809561 >> v2:172.21.15.187:3300/0 conn(0x7f10c8072760 0x7f10c8072bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 --2- 172.21.15.112:0/3920809561 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10c8145320 0x7f10c8145720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 --2- 172.21.15.112:0/3920809561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 0x7f10c80add70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 -- 172.21.15.112:0/3920809561 >> 172.21.15.112:0/3920809561 conn(0x7f10c80b8ac0 msgr2=0x7f10c80b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7fc256b1c640 1 -- 172.21.15.112:0/2337614767 shutdown_connections 2024-04-25T00:55:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7fc256b1c640 1 --2- 172.21.15.112:0/2337614767 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc2500af0e0 0x7fc250146d20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7fc256b1c640 1 --2- 172.21.15.112:0/2337614767 >> v2:172.21.15.187:3300/0 conn(0x7fc2500b4a70 0x7fc2500b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7fc256b1c640 1 -- 172.21.15.112:0/2337614767 >> 172.21.15.112:0/2337614767 conn(0x7fc2500b8ac0 msgr2=0x7fc2500b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f7585aa1640 1 Processor -- start 2024-04-25T00:55:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7fc256b1c640 1 -- 172.21.15.112:0/2337614767 shutdown_connections 2024-04-25T00:55:45.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7fc256b1c640 1 -- 172.21.15.112:0/2337614767 wait complete. 2024-04-25T00:55:45.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f316ffff640 1 --2- 172.21.15.112:0/329810591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3168036aa0 0x7f31680d4420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f315c0097f0 tx=0x7f315c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344b9eca501e3efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f316effd640 1 -- 172.21.15.112:0/329810591 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c0b0070 con 0x7f3168036aa0 2024-04-25T00:55:45.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f316effd640 1 -- 172.21.15.112:0/329810591 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315c0ab600 con 0x7f3168036aa0 2024-04-25T00:55:45.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f316effd640 1 -- 172.21.15.112:0/329810591 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c0ab9b0 con 0x7f3168036aa0 2024-04-25T00:55:45.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 -- 172.21.15.112:0/3920809561 shutdown_connections 2024-04-25T00:55:45.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 -- 172.21.15.112:0/2209043134 wait complete. 2024-04-25T00:55:45.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 Processor -- start 2024-04-25T00:55:45.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a116b5640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8003ba0 0x7f19f8003fa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1a04001fd0 tx=0x7f1a0401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c50308db80711be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f3175296640 1 -- 172.21.15.112:0/329810591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3168036aa0 msgr2=0x7f31680d4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f3175296640 1 --2- 172.21.15.112:0/329810591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3168036aa0 0x7f31680d4420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f315c0097f0 tx=0x7f315c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f3175296640 1 -- 172.21.15.112:0/329810591 shutdown_connections 2024-04-25T00:55:45.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f3175296640 1 --2- 172.21.15.112:0/329810591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3168036aa0 0x7f31680d4420 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f3175296640 1 -- 172.21.15.112:0/329810591 >> 172.21.15.112:0/329810591 conn(0x7f316803abd0 msgr2=0x7f3168005ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f3175296640 1 -- 172.21.15.112:0/329810591 shutdown_connections 2024-04-25T00:55:45.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f3175296640 1 -- 172.21.15.112:0/329810591 wait complete. 2024-04-25T00:55:45.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f10cd76a640 1 -- 172.21.15.112:0/3920809561 wait complete. 2024-04-25T00:55:45.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10cd76a640 1 Processor -- start 2024-04-25T00:55:45.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7fc256b1c640 1 Processor -- start 2024-04-25T00:55:45.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7fc256b1c640 1 -- start start 2024-04-25T00:55:45.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f59c3fff640 1 -- start start 2024-04-25T00:55:45.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 0x7f59c413c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c3fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c40b57d0 0x7f59c41461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59c41472a0 0x7f59c4144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.188 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c3fff640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f59c40b5e50 con 0x7f59c41472a0 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c3fff640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f59c414fbb0 con 0x7f59c40b57d0 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c3fff640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59c4137d50 con 0x7f59c40728a0 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f7585aa1640 1 -- start start 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f7585aa1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7578038840 0x7f75780cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f7585aa1640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75780ccc60 0x7f757803ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f7585aa1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7578039350 con 0x7f757803b070 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f7585aa1640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7578038c20 con 0x7f75780ccc60 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f7585aa1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75780395f0 con 0x7f7578038840 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f757ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75780ccc60 0x7f757803ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.160+0000 7f7584a9f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7578038840 0x7f75780cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f757ffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f75780ccc60 0x7f757803ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40852/0 (socket says 172.21.15.112:40852) 2024-04-25T00:55:45.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f757ffff640 1 -- 172.21.15.112:0/4266743503 learned_addr learned my addr 172.21.15.112:0/4266743503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9db2b640 1 --2- 172.21.15.112:0/4226184557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 0x7f9e9814f9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009870 tx=0x7f9e8801c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5420cc3399fabe47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f7584a9f640 1 -- 172.21.15.112:0/4266743503 >> v2:172.21.15.187:3300/0 conn(0x7f75780ccc60 msgr2=0x7f757803ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f7584a9f640 1 --2- 172.21.15.112:0/4266743503 >> v2:172.21.15.187:3300/0 conn(0x7f75780ccc60 0x7f757803ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f7584a9f640 1 -- 172.21.15.112:0/4266743503 >> v1:172.21.15.187:6789/0 conn(0x7f757803b070 legacy=0x7f7578037500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3175296640 1 Processor -- start 2024-04-25T00:55:45.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10cd76a640 1 -- start start 2024-04-25T00:55:45.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10cd76a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10c8072760 0x7f10c8141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10cd76a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 0x7f10c81e7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10cd76a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8145320 0x7f10c81e7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10cd76a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f10c80b1f90 con 0x7f10c80b48c0 2024-04-25T00:55:45.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10cd76a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f10c80afcd0 con 0x7f10c8145320 2024-04-25T00:55:45.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc256b1c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2500af0e0 0x7fc25009e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc256b1c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2500b4a70 0x7fc250143f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc256b1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc250147260 0x7fc25013ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc256b1c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc2500ae230 con 0x7fc250147260 2024-04-25T00:55:45.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc256b1c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc2500af5b0 con 0x7fc2500af0e0 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc256b1c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc2500ae3b0 con 0x7fc2500b4a70 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc250147260 0x7fc25013ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc250147260 0x7fc25013ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51114/0 (socket says 172.21.15.112:51114) 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 -- 172.21.15.112:0/1451037271 learned_addr learned my addr 172.21.15.112:0/1451037271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2548b8640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2500af0e0 0x7fc25009e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc24ffff640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2500b4a70 0x7fc250143f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 -- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2500b4a70 msgr2=0x7fc250143f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc2500b4a70 0x7fc250143f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 -- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2500af0e0 msgr2=0x7fc25009e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc2500af0e0 0x7fc25009e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 -- 172.21.15.112:0/1451037271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25009dd70 con 0x7fc250147260 2024-04-25T00:55:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c40b57d0 0x7f59c41461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c40b57d0 0x7f59c41461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40868/0 (socket says 172.21.15.112:40868) 2024-04-25T00:55:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 -- 172.21.15.112:0/972257630 learned_addr learned my addr 172.21.15.112:0/972257630 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c2ffd640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 0x7f59c413c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 -- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 msgr2=0x7f59c413c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f59c40728a0 0x7f59c413c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 -- 172.21.15.112:0/972257630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59c41472a0 msgr2=0x7f59c4144440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59c41472a0 0x7f59c4144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 -- 172.21.15.112:0/972257630 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c4139960 con 0x7f59c40b57d0 2024-04-25T00:55:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c27fc640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c40b57d0 0x7f59c41461e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59b801a0f0 tx=0x7f59b8019260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f7584a9f640 1 -- 172.21.15.112:0/4266743503 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75780398d0 con 0x7f7578038840 2024-04-25T00:55:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e8ffff640 1 -- 172.21.15.112:0/4226184557 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e880b0070 con 0x7f9e980b6610 2024-04-25T00:55:45.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e8ffff640 1 -- 172.21.15.112:0/4226184557 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e880ab600 con 0x7f9e980b6610 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e8ffff640 1 -- 172.21.15.112:0/4226184557 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e880ab9b0 con 0x7f9e980b6610 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 -- 172.21.15.112:0/4226184557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 msgr2=0x7f9e9814f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 --2- 172.21.15.112:0/4226184557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 0x7f9e9814f9c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009870 tx=0x7f9e8801c490 comp rx=0 tx=0).stop 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 -- 172.21.15.112:0/4226184557 shutdown_connections 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 --2- 172.21.15.112:0/4226184557 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 0x7f9e9814f9c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 --2- 172.21.15.112:0/4226184557 >> v2:172.21.15.112:3300/0 conn(0x7f9e980b73e0 0x7f9e98145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 -- 172.21.15.112:0/4226184557 >> 172.21.15.112:0/4226184557 conn(0x7f9e9806ee00 msgr2=0x7f9e9806f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 -- 172.21.15.112:0/4226184557 shutdown_connections 2024-04-25T00:55:45.196 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04001c20 con 0x7f19f8003ba0 2024-04-25T00:55:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a04003ba0 con 0x7f19f8003ba0 2024-04-25T00:55:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a03fff640 1 -- 172.21.15.112:0/479107762 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04005080 con 0x7f19f8003ba0 2024-04-25T00:55:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a13919640 1 -- 172.21.15.112:0/479107762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8003ba0 msgr2=0x7f19f8003fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a13919640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8003ba0 0x7f19f8003fa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1a04001fd0 tx=0x7f1a0401c920 comp rx=0 tx=0).stop 2024-04-25T00:55:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3175296640 1 -- start start 2024-04-25T00:55:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3175296640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168036aa0 0x7f31680cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3175296640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31680cab10 0x7f31680d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 -- 172.21.15.112:0/4226184557 wait complete. 2024-04-25T00:55:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc2550b9640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc250147260 0x7fc25013ec80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c730 tx=0x7fc24800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.198 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a13919640 1 -- 172.21.15.112:0/479107762 shutdown_connections 2024-04-25T00:55:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a13919640 1 --2- 172.21.15.112:0/479107762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19f8003ba0 0x7f19f8003fa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a13919640 1 -- 172.21.15.112:0/479107762 >> 172.21.15.112:0/479107762 conn(0x7f1a0c0b8d30 msgr2=0x7f1a0c0b9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a13919640 1 -- 172.21.15.112:0/479107762 shutdown_connections 2024-04-25T00:55:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3175296640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31680d10b0 0x7f31680d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3175296640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f31680df650 con 0x7f3168036aa0 2024-04-25T00:55:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3175296640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f31680d4960 con 0x7f31680cab10 2024-04-25T00:55:45.199 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3175296640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f31680df7d0 con 0x7f31680d10b0 2024-04-25T00:55:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10cd76a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10c80b2dc0 con 0x7f10c8072760 2024-04-25T00:55:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8145320 0x7f10c81e7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8145320 0x7f10c81e7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40892/0 (socket says 172.21.15.112:40892) 2024-04-25T00:55:45.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c77fe640 1 -- 172.21.15.112:0/2728790191 learned_addr learned my addr 172.21.15.112:0/2728790191 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c67fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 0x7f10c81e7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248016020 con 0x7fc250147260 2024-04-25T00:55:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc248006d60 con 0x7fc250147260 2024-04-25T00:55:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248003bc0 con 0x7fc250147260 2024-04-25T00:55:45.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f316f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31680cab10 0x7f31680d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3174a95640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31680d10b0 0x7f31680d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f316f7fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31680cab10 0x7f31680d8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40898/0 (socket says 172.21.15.112:40898) 2024-04-25T00:55:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f316f7fe640 1 -- 172.21.15.112:0/3399420707 learned_addr learned my addr 172.21.15.112:0/3399420707 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3174a95640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31680d10b0 0x7f31680d91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51310/0 (socket says 172.21.15.112:51310) 2024-04-25T00:55:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b80b0040 con 0x7f59c40b57d0 2024-04-25T00:55:45.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b80bc070 con 0x7f59c40b57d0 2024-04-25T00:55:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c4137fa0 con 0x7f59c40b57d0 2024-04-25T00:55:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b80b6890 con 0x7f59c40b57d0 2024-04-25T00:55:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc250145b30 con 0x7fc250147260 2024-04-25T00:55:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25009b5f0 con 0x7fc250147260 2024-04-25T00:55:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c67fc640 1 -- 172.21.15.112:0/2728790191 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10c8072760 msgr2=0x7f10c8141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c67fc640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f10c8072760 0x7f10c8141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c67fc640 1 -- 172.21.15.112:0/2728790191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8145320 msgr2=0x7f10c81e7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c67fc640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f10c8145320 0x7f10c81e7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f10c67fc640 1 -- 172.21.15.112:0/2728790191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c8151d20 con 0x7f10c80b48c0 2024-04-25T00:55:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3174a95640 1 -- 172.21.15.112:0/3399420707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31680cab10 msgr2=0x7f31680d8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3174a95640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f31680cab10 0x7f31680d8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3174a95640 1 -- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168036aa0 msgr2=0x7f31680cdc50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:55:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3174a95640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3168036aa0 0x7f31680cdc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.204 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3174a95640 1 -- 172.21.15.112:0/3399420707 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31680d98d0 con 0x7f31680d10b0 2024-04-25T00:55:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 Processor -- start 2024-04-25T00:55:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a13919640 1 -- 172.21.15.112:0/479107762 wait complete. 2024-04-25T00:55:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f1a13919640 1 Processor -- start 2024-04-25T00:55:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f3174a95640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31680d10b0 0x7f31680d91f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f317006b180 tx=0x7f3170068e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.205 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c413cbe0 con 0x7f59c40b57d0 2024-04-25T00:55:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.164+0000 7f9e9f58e640 1 -- start start 2024-04-25T00:55:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59c404f510 con 0x7f59c40b57d0 2024-04-25T00:55:45.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a13919640 1 -- start start 2024-04-25T00:55:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a13919640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8003ba0 0x7f1a0c1418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a13919640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a0c147ba0 0x7f1a0c13e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a13919640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0c13ec80 0x7f1a0c148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a13919640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b1240 con 0x7f1a0c13ec80 2024-04-25T00:55:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a13919640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c0b0040 con 0x7f19f8003ba0 2024-04-25T00:55:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a13919640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c06c100 con 0x7f1a0c147ba0 2024-04-25T00:55:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0c13ec80 0x7f1a0c148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0c13ec80 0x7f1a0c148420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51158/0 (socket says 172.21.15.112:51158) 2024-04-25T00:55:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 -- 172.21.15.112:0/978811659 learned_addr learned my addr 172.21.15.112:0/978811659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a116b5640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8003ba0 0x7f1a0c1418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317008f070 con 0x7f31680d10b0 2024-04-25T00:55:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31680d73f0 con 0x7f31680d10b0 2024-04-25T00:55:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f316816ca90 con 0x7f31680d10b0 2024-04-25T00:55:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f10c67fc640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 0x7f10c81e7900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00e8e0 tx=0x7f10bc00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc25004f580 con 0x7fc250147260 2024-04-25T00:55:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7584a9f640 1 --2- 172.21.15.112:0/4266743503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7578038840 0x7f75780cc720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f75740097c0 tx=0x7f757401c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7022c2843652d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f757f7fe640 1 -- 172.21.15.112:0/4266743503 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740b0070 con 0x7f7578038840 2024-04-25T00:55:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f757f7fe640 1 -- 172.21.15.112:0/4266743503 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75740ab600 con 0x7f7578038840 2024-04-25T00:55:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f757f7fe640 1 -- 172.21.15.112:0/4266743503 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740ab9d0 con 0x7f7578038840 2024-04-25T00:55:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9f58e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e980b5d90 0x7f9e981406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9f58e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 0x7f9e98142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9f58e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e980b73e0 0x7f9e981384d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9f58e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e980b4d20 con 0x7f9e980b5d90 2024-04-25T00:55:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9f58e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9e980b4ba0 con 0x7f9e980b6610 2024-04-25T00:55:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9f58e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e9806b210 con 0x7f9e980b73e0 2024-04-25T00:55:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 -- 172.21.15.112:0/978811659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a0c147ba0 msgr2=0x7f1a0c13e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1a0c147ba0 0x7f1a0c13e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 -- 172.21.15.112:0/978811659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8003ba0 msgr2=0x7f1a0c1418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f19f8003ba0 0x7f1a0c1418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 -- 172.21.15.112:0/978811659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c13cce0 con 0x7f1a0c13ec80 2024-04-25T00:55:45.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f1a11eb6640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0c13ec80 0x7f1a0c148420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0751f0 tx=0x7f19fc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7585aa1640 1 -- 172.21.15.112:0/4266743503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7578038840 msgr2=0x7f75780cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7585aa1640 1 --2- 172.21.15.112:0/4266743503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7578038840 0x7f75780cc720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f75740097c0 tx=0x7f757401c3d0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc017070 con 0x7f10c80b48c0 2024-04-25T00:55:45.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10bc00c8e0 con 0x7f10c80b48c0 2024-04-25T00:55:45.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc00ccb0 con 0x7f10c80b48c0 2024-04-25T00:55:45.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c8142a00 con 0x7f10c80b48c0 2024-04-25T00:55:45.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc2480050d0 con 0x7fc250147260 2024-04-25T00:55:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7fc24d7fa640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc23c0678c0 0x7fc23c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc248086570 con 0x7fc250147260 2024-04-25T00:55:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7fc24d7fa640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fc23c06fb40 0x7fc23c071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 --> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc248024e50 con 0x7fc23c06fb40 2024-04-25T00:55:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc248086960 con 0x7fc250147260 2024-04-25T00:55:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7585aa1640 1 -- 172.21.15.112:0/4266743503 shutdown_connections 2024-04-25T00:55:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7585aa1640 1 --2- 172.21.15.112:0/4266743503 >> v2:172.21.15.187:3300/0 conn(0x7f75780ccc60 0x7f757803ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7585aa1640 1 --2- 172.21.15.112:0/4266743503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7578038840 0x7f75780cc720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7585aa1640 1 -- 172.21.15.112:0/4266743503 >> 172.21.15.112:0/4266743503 conn(0x7f757803ea70 msgr2=0x7f757803ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f59b80ab4c0 con 0x7f59c40b57d0 2024-04-25T00:55:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9d32a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e980b5d90 0x7f9e981406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9d32a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e980b5d90 0x7f9e981406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51146/0 (socket says 172.21.15.112:51146) 2024-04-25T00:55:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9d32a640 1 -- 172.21.15.112:0/3394834066 learned_addr learned my addr 172.21.15.112:0/3394834066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9cb29640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 0x7f9e98142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c8148450 con 0x7f10c80b48c0 2024-04-25T00:55:45.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7585aa1640 1 -- 172.21.15.112:0/4266743503 shutdown_connections 2024-04-25T00:55:45.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9db2b640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e980b73e0 0x7f9e981384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3170073820 con 0x7f31680d10b0 2024-04-25T00:55:45.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31700935f0 con 0x7f31680d10b0 2024-04-25T00:55:45.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f31680053a0 con 0x7f31680d10b0 2024-04-25T00:55:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f59ab7fe640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59b0067990 0x7f59b0069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f59b8120d90 con 0x7f59c40b57d0 2024-04-25T00:55:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f7585aa1640 1 -- 172.21.15.112:0/4266743503 wait complete. 2024-04-25T00:55:45.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9cb29640 1 -- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e980b73e0 msgr2=0x7f9e981384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9cb29640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e980b73e0 0x7f9e981384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9cb29640 1 -- 172.21.15.112:0/3394834066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e980b5d90 msgr2=0x7f9e981406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9cb29640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e980b5d90 0x7f9e981406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9cb29640 1 -- 172.21.15.112:0/3394834066 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e98142b40 con 0x7f9e980b6610 2024-04-25T00:55:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f59ab7fe640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7f59b006fbb0 0x7f59b0071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 --> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59b80c6e50 con 0x7f59b006fbb0 2024-04-25T00:55:45.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f59b8121180 con 0x7f59c40b57d0 2024-04-25T00:55:45.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7fc2548b8640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc23c0678c0 0x7fc23c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f9e9cb29640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 0x7f9e98142600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c400 tx=0x7f9e9400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f59c37fe640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7f59b006fbb0 0x7f59b0071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f59c2ffd640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59b0067990 0x7f59b0069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 Processor -- start 2024-04-25T00:55:45.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 -- start start 2024-04-25T00:55:45.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94017070 con 0x7f9e980b6610 2024-04-25T00:55:45.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3170092070 con 0x7f31680d10b0 2024-04-25T00:55:45.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f31540678f0 0x7f3154069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3170101a70 con 0x7f31680d10b0 2024-04-25T00:55:45.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f315406fb70 0x7f3154071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 --> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3154072640 con 0x7f315406fb70 2024-04-25T00:55:45.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.168+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3170101e60 con 0x7f31680d10b0 2024-04-25T00:55:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7fc2548b8640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc23c0678c0 0x7fc23c069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc244014f90 tx=0x7fc244008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7fc24ffff640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fc23c06fb40 0x7fc23c071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e981d75d0 con 0x7f9e980b6610 2024-04-25T00:55:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e98137fb0 con 0x7f9e980b6610 2024-04-25T00:55:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e940052d0 con 0x7f9e980b6610 2024-04-25T00:55:45.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94003840 con 0x7f9e980b6610 2024-04-25T00:55:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10c81410e0 con 0x7f10c80b48c0 2024-04-25T00:55:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f206f1f8640 1 Processor -- start 2024-04-25T00:55:45.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f206f1f8640 1 -- start start 2024-04-25T00:55:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f59c2ffd640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59b0067990 0x7f59b0069e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59b40059e0 tx=0x7f59b400d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f757803b070 0x7f7578036de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75780ccc60 0x7f7578169bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75780c3ef0 0x7f757816d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f75780d97e0 con 0x7f75780ccc60 2024-04-25T00:55:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7578037e20 con 0x7f757803b070 2024-04-25T00:55:45.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f75780d9960 con 0x7f75780c3ef0 2024-04-25T00:55:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f59c37fe640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7f59b006fbb0 0x7f59b0071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f316f7fe640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f315406fb70 0x7f3154071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9e9804f510 con 0x7f9e980b6610 2024-04-25T00:55:45.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f757803b070 0x7f7578036de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f316ffff640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f31540678f0 0x7f3154069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f757803b070 0x7f7578036de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40926/0 (socket says 172.21.15.112:40926) 2024-04-25T00:55:45.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f75852a0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75780c3ef0 0x7f757816d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 -- 172.21.15.112:0/564867975 learned_addr learned my addr 172.21.15.112:0/564867975 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7fc24ffff640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fc23c06fb40 0x7fc23c071f60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f316f7fe640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f315406fb70 0x7f3154071f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f316ffff640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f31540678f0 0x7f3154069db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f315c0ad000 tx=0x7f315c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 -- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75780c3ef0 msgr2=0x7f757816d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f75780c3ef0 0x7f757816d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 -- 172.21.15.112:0/564867975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75780ccc60 msgr2=0x7f7578169bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75780ccc60 0x7f7578169bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 -- 172.21.15.112:0/564867975 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75780cf640 con 0x7f757803b070 2024-04-25T00:55:45.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc017070 con 0x7f1a0c13ec80 2024-04-25T00:55:45.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc00c620 con 0x7f1a0c13ec80 2024-04-25T00:55:45.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00c9f0 con 0x7f1a0c13ec80 2024-04-25T00:55:45.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f206f1f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f206001dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0c146210 con 0x7f1a0c13ec80 2024-04-25T00:55:45.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0c1470c0 con 0x7f1a0c13ec80 2024-04-25T00:55:45.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7584a9f640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f757803b070 0x7f7578036de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75740ae040 tx=0x7f7574019170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f206f1f8640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20600cab30 0x7f20600caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 <== osd.2 v2:172.21.15.112:6819/3154807242 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3154072640 con 0x7f315406fb70 2024-04-25T00:55:45.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740b0040 con 0x7f757803b070 2024-04-25T00:55:45.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f206e1f6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f206001dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f206e1f6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f206001dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51348/0 (socket says 172.21.15.112:51348) 2024-04-25T00:55:45.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f206d9f5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20600cab30 0x7f20600caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f206d9f5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f20600cab30 0x7f20600caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40940/0 (socket says 172.21.15.112:40940) 2024-04-25T00:55:45.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f10bc01a070 con 0x7f10c80b48c0 2024-04-25T00:55:45.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10af7fe640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10b4067910 0x7f10b4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f10bc085cd0 con 0x7f10c80b48c0 2024-04-25T00:55:45.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10af7fe640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f10b406fb30 0x7f10b4071f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 --> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10bc014e50 con 0x7f10b406fb30 2024-04-25T00:55:45.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f10bc0861b0 con 0x7f10c80b48c0 2024-04-25T00:55:45.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a0c04f510 con 0x7f1a0c13ec80 2024-04-25T00:55:45.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10c6ffd640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10b4067910 0x7f10b4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f10c77fe640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f10b406fb30 0x7f10b4071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75740ab400 con 0x7f757803b070 2024-04-25T00:55:45.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7574006e20 con 0x7f757803b070 2024-04-25T00:55:45.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75780d07c0 con 0x7f757803b070 2024-04-25T00:55:45.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75780d04e0 con 0x7f757803b070 2024-04-25T00:55:45.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75780c4f60 con 0x7f757803b070 2024-04-25T00:55:45.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f206e1f6640 1 -- 172.21.15.112:0/2350435969 learned_addr learned my addr 172.21.15.112:0/2350435969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f206f1f8640 1 -- 172.21.15.112:0/2350435969 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20600336a0 con 0x7f206003a770 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f206f1f8640 1 -- 172.21.15.112:0/2350435969 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f20600cb4d0 con 0x7f20600cab30 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f206f1f8640 1 -- 172.21.15.112:0/2350435969 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f206001e830 con 0x7f206000c530 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f206e1f6640 1 -- 172.21.15.112:0/2350435969 >> v2:172.21.15.187:3300/0 conn(0x7f20600cab30 msgr2=0x7f20600caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f206e1f6640 1 --2- 172.21.15.112:0/2350435969 >> v2:172.21.15.187:3300/0 conn(0x7f20600cab30 0x7f20600caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f206e1f6640 1 -- 172.21.15.112:0/2350435969 >> v1:172.21.15.112:6789/0 conn(0x7f206003a770 legacy=0x7f206003abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f206e1f6640 1 -- 172.21.15.112:0/2350435969 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206001e1e0 con 0x7f206000c530 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f206e1f6640 1 --2- 172.21.15.112:0/2350435969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f206001dbc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2068062db0 tx=0x7f2068097ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=727b5c4f37e71b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f19fc01a070 con 0x7f1a0c13ec80 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f1a01ffb640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f19f00679c0 0x7f19f0069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f19fc085e00 con 0x7f1a0c13ec80 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f1a01ffb640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7f19f006fc40 0x7f19f0072060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 --> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19fc014e50 con 0x7f19f006fc40 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f19fc0861f0 con 0x7f1a0c13ec80 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f1a116b5640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f19f00679c0 0x7f19f0069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f1a10eb4640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7f19f006fc40 0x7f19f0072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f1a10eb4640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7f19f006fc40 0x7f19f0072060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f1a116b5640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f19f00679c0 0x7f19f0069e80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005000 tx=0x7f1a040a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f10c77fe640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f10b406fb30 0x7f10b4071f50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f10c6ffd640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10b4067910 0x7f10b4069dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10b80ad000 tx=0x7f10b8004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 <== osd.3 v2:172.21.15.112:6800/3692662359 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10bc014e50 con 0x7f10b406fb30 2024-04-25T00:55:45.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 <== osd.4 v2:172.21.15.187:6801/1453523944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc248024e50 con 0x7fc23c06fb40 2024-04-25T00:55:45.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 <== osd.5 v2:172.21.15.187:6816/2456695186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59b80c6e50 con 0x7f59b006fbb0 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f75740bb020 con 0x7f757803b070 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f757d7fa640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f75640678c0 0x7f7564069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f757411fe90 con 0x7f757803b070 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f757d7fa640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7f756406fb40 0x7f7564071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 --> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75740bfe80 con 0x7f756406fb40 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7574120280 con 0x7f757803b070 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f75852a0640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7f756406fb40 0x7f7564071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f757ffff640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f75640678c0 0x7f7564069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f75852a0640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7f756406fb40 0x7f7564071f60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 <== mon.1 v2:172.21.15.187:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e9401a070 con 0x7f9e980b6610 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.172+0000 7f9e8dffb640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e7c0678c0 0x7f9e7c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f9e9d32a640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e7c0678c0 0x7f9e7c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 <== mon.1 v2:172.21.15.187:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9e94085700 con 0x7f9e980b6610 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f9e8dffb640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9e7c06fac0 0x7f9e7c071ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 --> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9e94020ec0 con 0x7f9e7c06fac0 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 <== mon.1 v2:172.21.15.187:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9e94085ab0 con 0x7f9e980b6610 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f9e9db2b640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9e7c06fac0 0x7f9e7c071ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f9e9d32a640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e7c0678c0 0x7f9e7c069d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009f50 tx=0x7f9e900095b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.176+0000 7f9e9db2b640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9e7c06fac0 0x7f9e7c071ee0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 <== osd.1 v2:172.21.15.112:6816/3975585121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19fc014e50 con 0x7f19f006fc40 2024-04-25T00:55:45.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.180+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 <== osd.6 v2:172.21.15.187:6800/718817690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75740bfe80 con 0x7f756406fb40 2024-04-25T00:55:45.244 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.184+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 <== osd.7 v2:172.21.15.187:6824/2342593789 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9e94020ec0 con 0x7f9e7c06fac0 2024-04-25T00:55:45.245 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.184+0000 7f757ffff640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f75640678c0 0x7f7564069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f756c006f50 tx=0x7f756c006ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.184+0000 7f206d1f4640 1 -- 172.21.15.112:0/2350435969 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206812d070 con 0x7f206000c530 2024-04-25T00:55:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.184+0000 7f206d1f4640 1 -- 172.21.15.112:0/2350435969 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2068120e30 con 0x7f206000c530 2024-04-25T00:55:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.184+0000 7f206d1f4640 1 -- 172.21.15.112:0/2350435969 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068128640 con 0x7f206000c530 2024-04-25T00:55:45.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 -- 172.21.15.112:0/2350435969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 msgr2=0x7f206001dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 --2- 172.21.15.112:0/2350435969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f206001dbc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2068062db0 tx=0x7f2068097ec0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 -- 172.21.15.112:0/2350435969 shutdown_connections 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 --2- 172.21.15.112:0/2350435969 >> v2:172.21.15.187:3300/0 conn(0x7f20600cab30 0x7f20600caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 --2- 172.21.15.112:0/2350435969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f206001dbc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 -- 172.21.15.112:0/2350435969 >> 172.21.15.112:0/2350435969 conn(0x7f206000b7b0 msgr2=0x7f206000bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 -- 172.21.15.112:0/2350435969 shutdown_connections 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 -- 172.21.15.112:0/2350435969 wait complete. 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 Processor -- start 2024-04-25T00:55:45.248 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.188+0000 7f206f1f8640 1 -- start start 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206f1f8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f2060166ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206f1f8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206003a770 0x7f20600c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206f1f8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20600cab30 0x7f2060035b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206e1f6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f2060166ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206e1f6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f2060166ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51364/0 (socket says 172.21.15.112:51364) 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f207045a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20600cab30 0x7f2060035b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f207045a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20600cab30 0x7f2060035b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51184/0 (socket says 172.21.15.112:51184) 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206d9f5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206003a770 0x7f20600c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206d9f5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206003a770 0x7f20600c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40950/0 (socket says 172.21.15.112:40950) 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206e1f6640 1 -- 172.21.15.112:0/164423430 learned_addr learned my addr 172.21.15.112:0/164423430 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f20600d7740 con 0x7f20600cab30 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f20600d75c0 con 0x7f206003a770 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f20600d78c0 con 0x7f206000c530 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f207045a640 1 -- 172.21.15.112:0/164423430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 msgr2=0x7f2060166ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f207045a640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f206000c530 0x7f2060166ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f207045a640 1 -- 172.21.15.112:0/164423430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206003a770 msgr2=0x7f20600c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f207045a640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f206003a770 0x7f20600c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f207045a640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2060039980 con 0x7f20600cab30 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f207045a640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20600cab30 0x7f2060035b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f206400c440 tx=0x7f206400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064017070 con 0x7f20600cab30 2024-04-25T00:55:45.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 --> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59c40afad0 con 0x7f59b006fbb0 2024-04-25T00:55:45.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2064003600 con 0x7f20600cab30 2024-04-25T00:55:45.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2060166ff0 con 0x7f20600cab30 2024-04-25T00:55:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.192+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20601662b0 con 0x7f20600cab30 2024-04-25T00:55:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f59ab7fe640 1 -- 172.21.15.112:0/972257630 <== osd.5 v2:172.21.15.187:6816/2456695186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f59c40afad0 con 0x7f59b006fbb0 2024-04-25T00:55:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20640039f0 con 0x7f20600cab30 2024-04-25T00:55:45.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2050001040 con 0x7f20600cab30 2024-04-25T00:55:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7f59b006fbb0 msgr2=0x7f59b0071fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f59c3fff640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7f59b006fbb0 0x7f59b0071fd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59b0067990 msgr2=0x7f59b0069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f59c3fff640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59b0067990 0x7f59b0069e50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59b40059e0 tx=0x7f59b400d040 comp rx=0 tx=0).stop 2024-04-25T00:55:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c40b57d0 msgr2=0x7f59c41461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f59c3fff640 1 --2- 172.21.15.112:0/972257630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f59c40b57d0 0x7f59c41461e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f59b801a0f0 tx=0x7f59b8019260 comp rx=0 tx=0).stop 2024-04-25T00:55:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f59c37fe640 1 -- 172.21.15.112:0/972257630 reap_dead start 2024-04-25T00:55:45.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f206401a070 con 0x7f20600cab30 2024-04-25T00:55:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f205effd640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2054067af0 0x7f2054069fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2064084fb0 con 0x7f20600cab30 2024-04-25T00:55:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f205effd640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f205406fd70 0x7f2054072190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f206e1f6640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2054067af0 0x7f2054069fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2064003600 con 0x7f205406fd70 2024-04-25T00:55:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2064087610 con 0x7f20600cab30 2024-04-25T00:55:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f206d9f5640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f205406fd70 0x7f2054072190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.196+0000 7f206e1f6640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2054067af0 0x7f2054069fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2068050360 tx=0x7f206812a040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 shutdown_connections 2024-04-25T00:55:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 >> 172.21.15.112:0/972257630 conn(0x7f59c406c0a0 msgr2=0x7f59c40b9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f206d9f5640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f205406fd70 0x7f2054072190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 <== osd.0 v2:172.21.15.112:6804/1219565350 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2064003600 con 0x7f205406fd70 2024-04-25T00:55:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 --> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc2500b21d0 con 0x7fc23c06fb40 2024-04-25T00:55:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 --> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f10c804f580 con 0x7f10b406fb30 2024-04-25T00:55:45.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10af7fe640 1 -- 172.21.15.112:0/2728790191 <== osd.3 v2:172.21.15.112:6800/3692662359 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f10c804f580 con 0x7f10b406fb30 2024-04-25T00:55:45.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7fc24d7fa640 1 -- 172.21.15.112:0/1451037271 <== osd.4 v2:172.21.15.187:6801/1453523944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc2500b21d0 con 0x7fc23c06fb40 2024-04-25T00:55:45.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 --> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f316803ba70 con 0x7f315406fb70 2024-04-25T00:55:45.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fc23c06fb40 msgr2=0x7fc23c071f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7fc256b1c640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fc23c06fb40 0x7fc23c071f60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f10b406fb30 msgr2=0x7f10b4071f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10cd76a640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f10b406fb30 0x7f10b4071f50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f316cff9640 1 -- 172.21.15.112:0/3399420707 <== osd.2 v2:172.21.15.112:6819/3154807242 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f316803ba70 con 0x7f315406fb70 2024-04-25T00:55:45.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10b4067910 msgr2=0x7f10b4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10cd76a640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10b4067910 0x7f10b4069dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f10b80ad000 tx=0x7f10b8004740 comp rx=0 tx=0).stop 2024-04-25T00:55:45.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 msgr2=0x7f10c81e7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10cd76a640 1 --2- 172.21.15.112:0/2728790191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10c80b48c0 0x7f10c81e7900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00e8e0 tx=0x7f10bc00eda0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7f10c77fe640 1 -- 172.21.15.112:0/2728790191 reap_dead start 2024-04-25T00:55:45.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 shutdown_connections 2024-04-25T00:55:45.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 >> 172.21.15.112:0/2728790191 conn(0x7f10c80b8ac0 msgr2=0x7f10c8069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 shutdown_connections 2024-04-25T00:55:45.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 shutdown_connections 2024-04-25T00:55:45.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f10cd76a640 1 -- 172.21.15.112:0/2728790191 wait complete. 2024-04-25T00:55:45.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f59c3fff640 1 -- 172.21.15.112:0/972257630 wait complete. 2024-04-25T00:55:45.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f315406fb70 msgr2=0x7f3154071f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f315406fb70 0x7f3154071f90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f31540678f0 msgr2=0x7f3154069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f31540678f0 0x7f3154069db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f315c0ad000 tx=0x7f315c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:55:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31680d10b0 msgr2=0x7f31680d91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 --2- 172.21.15.112:0/3399420707 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f31680d10b0 0x7f31680d91f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f317006b180 tx=0x7f3170068e80 comp rx=0 tx=0).stop 2024-04-25T00:55:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3174a95640 1 -- 172.21.15.112:0/3399420707 reap_dead start 2024-04-25T00:55:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 shutdown_connections 2024-04-25T00:55:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 >> 172.21.15.112:0/3399420707 conn(0x7f316803abd0 msgr2=0x7f31680d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.200+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc23c0678c0 msgr2=0x7fc23c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7fc256b1c640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc23c0678c0 0x7fc23c069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc244014f90 tx=0x7fc244008040 comp rx=0 tx=0).stop 2024-04-25T00:55:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 shutdown_connections 2024-04-25T00:55:45.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7f3175296640 1 -- 172.21.15.112:0/3399420707 wait complete. 2024-04-25T00:55:45.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc250147260 msgr2=0x7fc25013ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7fc256b1c640 1 --2- 172.21.15.112:0/1451037271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc250147260 0x7fc25013ec80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc24800c730 tx=0x7fc24800cb30 comp rx=0 tx=0).stop 2024-04-25T00:55:45.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7fc2550b9640 1 -- 172.21.15.112:0/1451037271 reap_dead start 2024-04-25T00:55:45.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 shutdown_connections 2024-04-25T00:55:45.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 >> 172.21.15.112:0/1451037271 conn(0x7fc2500b8ac0 msgr2=0x7fc2500b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.204+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 shutdown_connections 2024-04-25T00:55:45.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.208+0000 7fc256b1c640 1 -- 172.21.15.112:0/1451037271 wait complete. 2024-04-25T00:55:45.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 --> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1a0c0b6990 con 0x7f19f006fc40 2024-04-25T00:55:45.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a01ffb640 1 -- 172.21.15.112:0/978811659 <== osd.1 v2:172.21.15.112:6816/3975585121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1a0c0b6990 con 0x7f19f006fc40 2024-04-25T00:55:45.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7f19f006fc40 msgr2=0x7f19f0072060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7f19f006fc40 0x7f19f0072060 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f19f00679c0 msgr2=0x7f19f0069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f19f00679c0 0x7f19f0069e80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005000 tx=0x7f1a040a7000 comp rx=0 tx=0).stop 2024-04-25T00:55:45.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0c13ec80 msgr2=0x7f1a0c148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 --2- 172.21.15.112:0/978811659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1a0c13ec80 0x7f1a0c148420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0751f0 tx=0x7f19fc00ec00 comp rx=0 tx=0).stop 2024-04-25T00:55:45.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a11eb6640 1 -- 172.21.15.112:0/978811659 reap_dead start 2024-04-25T00:55:45.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 shutdown_connections 2024-04-25T00:55:45.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 >> 172.21.15.112:0/978811659 conn(0x7f1a0c0b8d30 msgr2=0x7f1a0c14b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 shutdown_connections 2024-04-25T00:55:45.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.212+0000 7f1a13919640 1 -- 172.21.15.112:0/978811659 wait complete. 2024-04-25T00:55:45.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 --> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9e980b1520 con 0x7f9e7c06fac0 2024-04-25T00:55:45.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e8dffb640 1 -- 172.21.15.112:0/3394834066 <== osd.7 v2:172.21.15.187:6824/2342593789 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9e980b1520 con 0x7f9e7c06fac0 2024-04-25T00:55:45.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9e7c06fac0 msgr2=0x7f9e7c071ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9e7c06fac0 0x7f9e7c071ee0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e7c0678c0 msgr2=0x7f9e7c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e7c0678c0 0x7f9e7c069d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009f50 tx=0x7f9e900095b0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 msgr2=0x7f9e98142600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 --2- 172.21.15.112:0/3394834066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e980b6610 0x7f9e98142600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400c400 tx=0x7f9e9400c8c0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9db2b640 1 -- 172.21.15.112:0/3394834066 reap_dead start 2024-04-25T00:55:45.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 shutdown_connections 2024-04-25T00:55:45.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 >> 172.21.15.112:0/3394834066 conn(0x7f9e9806ee00 msgr2=0x7f9e980bb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 shutdown_connections 2024-04-25T00:55:45.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.216+0000 7f9e9f58e640 1 -- 172.21.15.112:0/3394834066 wait complete. 2024-04-25T00:55:45.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 --> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75780d6de0 con 0x7f756406fb40 2024-04-25T00:55:45.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f757d7fa640 1 -- 172.21.15.112:0/564867975 <== osd.6 v2:172.21.15.187:6800/718817690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f75780d6de0 con 0x7f756406fb40 2024-04-25T00:55:45.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7f756406fb40 msgr2=0x7f7564071f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7f756406fb40 0x7f7564071f60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f75640678c0 msgr2=0x7f7564069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f75640678c0 0x7f7564069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f756c006f50 tx=0x7f756c006ea0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f757803b070 msgr2=0x7f7578036de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 --2- 172.21.15.112:0/564867975 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f757803b070 0x7f7578036de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f75740ae040 tx=0x7f7574019170 comp rx=0 tx=0).stop 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f75852a0640 1 -- 172.21.15.112:0/564867975 reap_dead start 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 shutdown_connections 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 >> 172.21.15.112:0/564867975 conn(0x7f757803ea70 msgr2=0x7f757803ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 shutdown_connections 2024-04-25T00:55:45.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.224+0000 7f7585aa1640 1 -- 172.21.15.112:0/564867975 wait complete. 2024-04-25T00:55:45.274 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.275 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T00:55:45.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.232+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 --> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2050002de0 con 0x7f205406fd70 2024-04-25T00:55:45.276 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T00:55:45.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T00:55:45.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.232+0000 7f205effd640 1 -- 172.21.15.112:0/164423430 <== osd.0 v2:172.21.15.112:6804/1219565350 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2050002de0 con 0x7f205406fd70 2024-04-25T00:55:45.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.232+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f205406fd70 msgr2=0x7f2054072190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.232+0000 7f206f1f8640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f205406fd70 0x7f2054072190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2054067af0 msgr2=0x7f2054069fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f206f1f8640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2054067af0 0x7f2054069fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2068050360 tx=0x7f206812a040 comp rx=0 tx=0).stop 2024-04-25T00:55:45.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20600cab30 msgr2=0x7f2060035b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f206f1f8640 1 --2- 172.21.15.112:0/164423430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f20600cab30 0x7f2060035b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f206400c440 tx=0x7f206400c900 comp rx=0 tx=0).stop 2024-04-25T00:55:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f207045a640 1 -- 172.21.15.112:0/164423430 reap_dead start 2024-04-25T00:55:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 shutdown_connections 2024-04-25T00:55:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 >> 172.21.15.112:0/164423430 conn(0x7f206000b7b0 msgr2=0x7f206003dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 shutdown_connections 2024-04-25T00:55:45.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.236+0000 7f206f1f8640 1 -- 172.21.15.112:0/164423430 wait complete. 2024-04-25T00:55:45.279 INFO:teuthology.orchestra.run.smithi112.stdout:55834574853 2024-04-25T00:55:45.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T00:55:45.281 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.281 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T00:55:45.282 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.282 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T00:55:45.283 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T00:55:45.284 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T00:55:45.285 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.285 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T00:55:45.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f078b073640 1 Processor -- start 2024-04-25T00:55:45.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f078b073640 1 -- start start 2024-04-25T00:55:45.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f078b073640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f077c0ead40 0x7f077c037800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f078b073640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c038390 0x7f077c0f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f078b073640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f077c0e4840 con 0x7f077c035fc0 2024-04-25T00:55:45.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f078b073640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f077c037140 con 0x7f077c0ead40 2024-04-25T00:55:45.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f078b073640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f077c0ea8b0 con 0x7f077c038390 2024-04-25T00:55:45.488 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d907e640 1 Processor -- start 2024-04-25T00:55:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f0783fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f077c0ead40 0x7f077c037800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f0788e0f640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f077c035fc0 0x7f077c0e4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57282/0 (socket says 172.21.15.112:57282) 2024-04-25T00:55:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f0788e0f640 1 -- 172.21.15.112:0/2705221481 learned_addr learned my addr 172.21.15.112:0/2705221481 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f0789610640 1 --2- 172.21.15.112:0/2705221481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c038390 0x7f077c0f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f0783fff640 1 -- 172.21.15.112:0/2705221481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c038390 msgr2=0x7f077c0f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f0783fff640 1 --2- 172.21.15.112:0/2705221481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c038390 0x7f077c0f4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f0783fff640 1 -- 172.21.15.112:0/2705221481 >> v1:172.21.15.112:6789/0 conn(0x7f077c035fc0 legacy=0x7f077c0e4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f0783fff640 1 -- 172.21.15.112:0/2705221481 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c0387e0 con 0x7f077c0ead40 2024-04-25T00:55:45.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d907e640 1 -- start start 2024-04-25T00:55:45.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fa1c0a46640 1 Processor -- start 2024-04-25T00:55:45.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f9dbb3ea640 1 Processor -- start 2024-04-25T00:55:45.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f9dbb3ea640 1 -- start start 2024-04-25T00:55:45.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fa1c0a46640 1 -- start start 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d907e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f30d40baed0 0x7f30d40b9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d907e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 0x7f30d41457e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d907e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30d40bb2b0 con 0x7f30d4145d20 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d907e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f30d40b90c0 con 0x7f30d40baed0 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d907e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30d40b9940 con 0x7f30d40ba2d0 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f30d40baed0 0x7f30d40b9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f30d40baed0 0x7f30d40b9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51212/0 (socket says 172.21.15.112:51212) 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d2d9d640 1 -- 172.21.15.112:0/2680943962 learned_addr learned my addr 172.21.15.112:0/2680943962 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d2d9d640 1 -- 172.21.15.112:0/2680943962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 msgr2=0x7f30d41457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d2d9d640 1 --2- 172.21.15.112:0/2680943962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 0x7f30d41457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d2d9d640 1 -- 172.21.15.112:0/2680943962 >> v1:172.21.15.187:6789/0 conn(0x7f30d4145d20 legacy=0x7f30d414f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7f30d2d9d640 1 -- 172.21.15.112:0/2680943962 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d40b88c0 con 0x7f30d40baed0 2024-04-25T00:55:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fa1c0a46640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa1bc0afd70 0x7fa1bc0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fa1c0a46640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc0b5500 0x7fa1bc0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fa1c0a46640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1bc0b0710 con 0x7fa1bc06c7c0 2024-04-25T00:55:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fa1c0a46640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa1bc13d8f0 con 0x7fa1bc0afd70 2024-04-25T00:55:45.493 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fa1c0a46640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa1bc0b1170 con 0x7fa1bc0b5500 2024-04-25T00:55:45.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fbfecd5f640 1 Processor -- start 2024-04-25T00:55:45.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9dbb3ea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40ae3b0 0x7f9db40b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9dbb3ea640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9db40b2780 0x7f9db414e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.484+0000 7fa1bbfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc0b5500 0x7fa1bc0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fa1baffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa1bc0afd70 0x7fa1bc0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fa1baffd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa1bc0afd70 0x7fa1bc0b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40954/0 (socket says 172.21.15.112:40954) 2024-04-25T00:55:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fa1baffd640 1 -- 172.21.15.112:0/2818223752 learned_addr learned my addr 172.21.15.112:0/2818223752 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.495 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfecd5f640 1 -- start start 2024-04-25T00:55:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9dbb3ea640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db40b52f0 con 0x7f9db40adfd0 2024-04-25T00:55:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9dbb3ea640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f9db40af0b0 con 0x7f9db40b2780 2024-04-25T00:55:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9db9987640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9db40b2780 0x7f9db414e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9dbb3ea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9db40b0800 con 0x7f9db40ae3b0 2024-04-25T00:55:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9db9987640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9db40b2780 0x7f9db414e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51226/0 (socket says 172.21.15.112:51226) 2024-04-25T00:55:45.496 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9db9987640 1 -- 172.21.15.112:0/2018239727 learned_addr learned my addr 172.21.15.112:0/2018239727 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fa1baffd640 1 -- 172.21.15.112:0/2818223752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc0b5500 msgr2=0x7fa1bc0b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fa1baffd640 1 --2- 172.21.15.112:0/2818223752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc0b5500 0x7fa1bc0b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fa1baffd640 1 -- 172.21.15.112:0/2818223752 >> v1:172.21.15.187:6789/0 conn(0x7fa1bc06c7c0 legacy=0x7fa1bc147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fa1baffd640 1 -- 172.21.15.112:0/2818223752 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc1467a0 con 0x7fa1bc0afd70 2024-04-25T00:55:45.497 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb50865f640 1 Processor -- start 2024-04-25T00:55:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9db8985640 1 --2- 172.21.15.112:0/2018239727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40ae3b0 0x7f9db40b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9db9987640 1 -- 172.21.15.112:0/2018239727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40ae3b0 msgr2=0x7f9db40b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9db9987640 1 --2- 172.21.15.112:0/2018239727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40ae3b0 0x7f9db40b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9db9987640 1 -- 172.21.15.112:0/2018239727 >> v1:172.21.15.187:6789/0 conn(0x7f9db40adfd0 legacy=0x7f9db40ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.498 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f9db9987640 1 -- 172.21.15.112:0/2018239727 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db4143f50 con 0x7f9db40b2780 2024-04-25T00:55:45.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfecd5f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfe814f870 0x7fbfe80b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfecd5f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe80b0590 0x7fbfe80b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfecd5f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbfe814b650 0x7fbfe814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.499 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfecd5f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe814ec00 con 0x7fbfe814f870 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfe814f870 0x7fbfe80b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbfe814f870 0x7fbfe80b5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51242/0 (socket says 172.21.15.112:51242) 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe77fe640 1 -- 172.21.15.112:0/507886368 learned_addr learned my addr 172.21.15.112:0/507886368 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfecd5f640 1 -- 172.21.15.112:0/507886368 --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbfe80b26b0 con 0x7fbfe814b650 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfecd5f640 1 -- 172.21.15.112:0/507886368 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe80b2990 con 0x7fbfe80b0590 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe7fff640 1 --2- 172.21.15.112:0/507886368 >> v2:172.21.15.187:3300/0 conn(0x7fbfe814b650 0x7fbfe814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe6ffd640 1 --2- 172.21.15.112:0/507886368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe80b0590 0x7fbfe80b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe77fe640 1 -- 172.21.15.112:0/507886368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe80b0590 msgr2=0x7fbfe80b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe77fe640 1 --2- 172.21.15.112:0/507886368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe80b0590 0x7fbfe80b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe77fe640 1 -- 172.21.15.112:0/507886368 >> v2:172.21.15.187:3300/0 conn(0x7fbfe814b650 msgr2=0x7fbfe814bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe77fe640 1 --2- 172.21.15.112:0/507886368 >> v2:172.21.15.187:3300/0 conn(0x7fbfe814b650 0x7fbfe814bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.500 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe77fe640 1 -- 172.21.15.112:0/507886368 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe80b4480 con 0x7fbfe814f870 2024-04-25T00:55:45.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fbfe6ffd640 1 --2- 172.21.15.112:0/507886368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe80b0590 0x7fbfe80b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:45.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb50865f640 1 -- start start 2024-04-25T00:55:45.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb50865f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb5000af900 0x7fb50014efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb50865f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50014f500 0x7fb50014f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.501 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb50865f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5000b5290 con 0x7fb50006c520 2024-04-25T00:55:45.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb50865f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb5000add70 con 0x7fb5000af900 2024-04-25T00:55:45.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb50865f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb50006c900 con 0x7fb50014f500 2024-04-25T00:55:45.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb5063fb640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fb50006c520 0x7fb5000ae5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:52750/0 (socket says 172.21.15.112:52750) 2024-04-25T00:55:45.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb5063fb640 1 -- 172.21.15.112:0/38249275 learned_addr learned my addr 172.21.15.112:0/38249275 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:45.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb506bfc640 1 --2- 172.21.15.112:0/38249275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50014f500 0x7fb50014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.502 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d2d9d640 1 --2- 172.21.15.112:0/2680943962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 0x7f30d40b9d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30c40098a0 tx=0x7f30c401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ed2f61ebcf2f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d1d9b640 1 -- 172.21.15.112:0/2680943962 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40ab600 con 0x7f30d40baed0 2024-04-25T00:55:45.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d1d9b640 1 -- 172.21.15.112:0/2680943962 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30c40b0070 con 0x7f30d40baed0 2024-04-25T00:55:45.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d1d9b640 1 -- 172.21.15.112:0/2680943962 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40ab9c0 con 0x7f30d40baed0 2024-04-25T00:55:45.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb505bfa640 1 --2- 172.21.15.112:0/38249275 >> v2:172.21.15.112:3300/0 conn(0x7fb5000af900 0x7fb50014efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb5053f9640 1 -- 172.21.15.112:0/38249275 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091949524 0 0) 0x7fb5000b5290 con 0x7fb50006c520 2024-04-25T00:55:45.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb506bfc640 1 -- 172.21.15.112:0/38249275 >> v2:172.21.15.112:3300/0 conn(0x7fb5000af900 msgr2=0x7fb50014efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb506bfc640 1 --2- 172.21.15.112:0/38249275 >> v2:172.21.15.112:3300/0 conn(0x7fb5000af900 0x7fb50014efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb506bfc640 1 -- 172.21.15.112:0/38249275 >> v1:172.21.15.187:6789/0 conn(0x7fb50006c520 legacy=0x7fb5000ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb506bfc640 1 -- 172.21.15.112:0/38249275 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5000aff80 con 0x7fb50014f500 2024-04-25T00:55:45.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fb505bfa640 1 --2- 172.21.15.112:0/38249275 >> v2:172.21.15.112:3300/0 conn(0x7fb5000af900 0x7fb50014efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d907e640 1 -- 172.21.15.112:0/2680943962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 msgr2=0x7f30d40b9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d907e640 1 --2- 172.21.15.112:0/2680943962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 0x7f30d40b9d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30c40098a0 tx=0x7f30c401c440 comp rx=0 tx=0).stop 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d907e640 1 -- 172.21.15.112:0/2680943962 shutdown_connections 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d907e640 1 --2- 172.21.15.112:0/2680943962 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 0x7f30d41457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d907e640 1 --2- 172.21.15.112:0/2680943962 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 0x7f30d40b9d90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d907e640 1 -- 172.21.15.112:0/2680943962 >> 172.21.15.112:0/2680943962 conn(0x7f30d4067120 msgr2=0x7f30d4067530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d907e640 1 -- 172.21.15.112:0/2680943962 shutdown_connections 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f30d907e640 1 -- 172.21.15.112:0/2680943962 wait complete. 2024-04-25T00:55:45.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7fa1baffd640 1 --2- 172.21.15.112:0/2818223752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 0x7fa1bc0b48a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009870 tx=0x7fa1b001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fdc5ec7da351654 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f0783fff640 1 --2- 172.21.15.112:0/2705221481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0ead40 0x7f077c037800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f077800a960 tx=0x7f077801e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0c767d5c42acc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f07837fe640 1 -- 172.21.15.112:0/2705221481 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077800b040 con 0x7f077c0ead40 2024-04-25T00:55:45.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f07837fe640 1 -- 172.21.15.112:0/2705221481 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07780aa630 con 0x7f077c0ead40 2024-04-25T00:55:45.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f07837fe640 1 -- 172.21.15.112:0/2705221481 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07780aaa00 con 0x7f077c0ead40 2024-04-25T00:55:45.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f078b073640 1 -- 172.21.15.112:0/2705221481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0ead40 msgr2=0x7f077c037800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f078b073640 1 --2- 172.21.15.112:0/2705221481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0ead40 0x7f077c037800 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f077800a960 tx=0x7f077801e760 comp rx=0 tx=0).stop 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f078b073640 1 -- 172.21.15.112:0/2705221481 shutdown_connections 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f078b073640 1 --2- 172.21.15.112:0/2705221481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c038390 0x7f077c0f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f078b073640 1 --2- 172.21.15.112:0/2705221481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0ead40 0x7f077c037800 secure :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f077800a960 tx=0x7f077801e760 comp rx=0 tx=0).stop 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f078b073640 1 -- 172.21.15.112:0/2705221481 >> 172.21.15.112:0/2705221481 conn(0x7f077c03a860 msgr2=0x7f077c03ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f078b073640 1 -- 172.21.15.112:0/2705221481 shutdown_connections 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.488+0000 7f078b073640 1 -- 172.21.15.112:0/2705221481 wait complete. 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9db9987640 1 --2- 172.21.15.112:0/2018239727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40b2780 0x7f9db414e1d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009e80 tx=0x7f9dac01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fd95e373028196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.507 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f4cdad3c640 1 Processor -- start 2024-04-25T00:55:45.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1ba7fc640 1 -- 172.21.15.112:0/2818223752 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00ab650 con 0x7fa1bc0afd70 2024-04-25T00:55:45.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1ba7fc640 1 -- 172.21.15.112:0/2818223752 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b00b0070 con 0x7fa1bc0afd70 2024-04-25T00:55:45.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1ba7fc640 1 -- 172.21.15.112:0/2818223752 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00aba10 con 0x7fa1bc0afd70 2024-04-25T00:55:45.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f4cdad3c640 1 -- start start 2024-04-25T00:55:45.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f078b073640 1 Processor -- start 2024-04-25T00:55:45.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dabfff640 1 -- 172.21.15.112:0/2018239727 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0a7900 con 0x7f9db40b2780 2024-04-25T00:55:45.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dabfff640 1 -- 172.21.15.112:0/2018239727 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dac0a7aa0 con 0x7f9db40b2780 2024-04-25T00:55:45.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dabfff640 1 -- 172.21.15.112:0/2018239727 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0af3f0 con 0x7f9db40b2780 2024-04-25T00:55:45.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfe77fe640 1 --2- 172.21.15.112:0/507886368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe814f870 0x7fbfe80b5130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8073250 tx=0x7fbfdc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0bf2098c50de6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- 172.21.15.112:0/2818223752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 msgr2=0x7fa1bc0b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 --2- 172.21.15.112:0/2818223752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 0x7fa1bc0b48a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009870 tx=0x7fa1b001c540 comp rx=0 tx=0).stop 2024-04-25T00:55:45.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d907e640 1 Processor -- start 2024-04-25T00:55:45.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d907e640 1 -- start start 2024-04-25T00:55:45.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- 172.21.15.112:0/2818223752 shutdown_connections 2024-04-25T00:55:45.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 --2- 172.21.15.112:0/2818223752 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc0b5500 0x7fa1bc0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 --2- 172.21.15.112:0/2818223752 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 0x7fa1bc0b48a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- 172.21.15.112:0/2818223752 >> 172.21.15.112:0/2818223752 conn(0x7fa1bc06bca0 msgr2=0x7fa1bc06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- 172.21.15.112:0/2818223752 shutdown_connections 2024-04-25T00:55:45.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f078b073640 1 -- start start 2024-04-25T00:55:45.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f078b073640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c035fc0 0x7f077c0edb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- 172.21.15.112:0/2818223752 wait complete. 2024-04-25T00:55:45.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb506bfc640 1 --2- 172.21.15.112:0/38249275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50014f500 0x7fb50014f950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009d90 tx=0x7fb4ec01c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42784f17c7412e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2018239727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40b2780 msgr2=0x7f9db414e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 --2- 172.21.15.112:0/2018239727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40b2780 0x7f9db414e1d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009e80 tx=0x7f9dac01eba0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2018239727 shutdown_connections 2024-04-25T00:55:45.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 --2- 172.21.15.112:0/2018239727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40b2780 0x7f9db414e1d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 --2- 172.21.15.112:0/2018239727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40ae3b0 0x7f9db40b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2018239727 >> 172.21.15.112:0/2018239727 conn(0x7f9db40b5980 msgr2=0x7f9db40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2018239727 shutdown_connections 2024-04-25T00:55:45.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2018239727 wait complete. 2024-04-25T00:55:45.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f078b073640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f077c038390 0x7f077c0ee060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb5053f9640 1 -- 172.21.15.112:0/38249275 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0b2070 con 0x7fb50014f500 2024-04-25T00:55:45.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f078b073640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0eb6c0 0x7f077c0ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb5053f9640 1 -- 172.21.15.112:0/38249275 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4ec0a5820 con 0x7fb50014f500 2024-04-25T00:55:45.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfe67fc640 1 -- 172.21.15.112:0/507886368 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0a7070 con 0x7fbfe814f870 2024-04-25T00:55:45.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfe67fc640 1 -- 172.21.15.112:0/507886368 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfdc0b0070 con 0x7fbfe814f870 2024-04-25T00:55:45.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfe67fc640 1 -- 172.21.15.112:0/507886368 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc0ab890 con 0x7fbfe814f870 2024-04-25T00:55:45.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f0789610640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0eb6c0 0x7f077c0ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f0789610640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0eb6c0 0x7f077c0ebb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51274/0 (socket says 172.21.15.112:51274) 2024-04-25T00:55:45.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 Processor -- start 2024-04-25T00:55:45.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d907e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 0x7f30d414b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d907e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 0x7f30d41e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d907e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d4145d20 0x7f30d41e5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d907e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4158ed0 con 0x7f30d40baed0 2024-04-25T00:55:45.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d907e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4158d50 con 0x7f30d4145d20 2024-04-25T00:55:45.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d907e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f30d4159050 con 0x7f30d40ba2d0 2024-04-25T00:55:45.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 Processor -- start 2024-04-25T00:55:45.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d359e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d4145d20 0x7f30d41e5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d359e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d4145d20 0x7f30d41e5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40972/0 (socket says 172.21.15.112:40972) 2024-04-25T00:55:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d359e640 1 -- 172.21.15.112:0/3354809516 learned_addr learned my addr 172.21.15.112:0/3354809516 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d2d9d640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 0x7f30d414b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d259c640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 0x7f30d41e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 -- 172.21.15.112:0/507886368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe814f870 msgr2=0x7fbfe80b5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.517 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/507886368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe814f870 0x7fbfe80b5130 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8073250 tx=0x7fbfdc01c520 comp rx=0 tx=0).stop 2024-04-25T00:55:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- start start 2024-04-25T00:55:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc06c7c0 0x7fa1bc143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 0x7fa1bc0ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1bc0b5500 0x7fa1bc0ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb50865f640 1 -- 172.21.15.112:0/38249275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50014f500 msgr2=0x7fb50014f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb50865f640 1 --2- 172.21.15.112:0/38249275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50014f500 0x7fb50014f950 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec009d90 tx=0x7fb4ec01c850 comp rx=0 tx=0).stop 2024-04-25T00:55:45.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc14a6b0 con 0x7fa1bc0b5500 2024-04-25T00:55:45.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa1bc14a530 con 0x7fa1bc0afd70 2024-04-25T00:55:45.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1c0a46640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa1bc14a830 con 0x7fa1bc06c7c0 2024-04-25T00:55:45.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f0788e0f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c035fc0 0x7f077c0edb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f0788e0f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c035fc0 0x7f077c0edb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51424/0 (socket says 172.21.15.112:51424) 2024-04-25T00:55:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 -- 172.21.15.112:0/507886368 shutdown_connections 2024-04-25T00:55:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/507886368 >> v2:172.21.15.187:3300/0 conn(0x7fbfe814b650 0x7fbfe814bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/507886368 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe80b0590 0x7fbfe80b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/507886368 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe814f870 0x7fbfe80b5130 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 -- 172.21.15.112:0/507886368 >> 172.21.15.112:0/507886368 conn(0x7fbfe806bca0 msgr2=0x7fbfe806c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1bc0b5500 0x7fa1bc0ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1bbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1bc0b5500 0x7fa1bc0ab5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51288/0 (socket says 172.21.15.112:51288) 2024-04-25T00:55:45.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1bbfff640 1 -- 172.21.15.112:0/1074301067 learned_addr learned my addr 172.21.15.112:0/1074301067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 -- 172.21.15.112:0/507886368 shutdown_connections 2024-04-25T00:55:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1bb7fe640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc06c7c0 0x7fa1bc143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f9dbb3ea640 1 -- start start 2024-04-25T00:55:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d259c640 1 -- 172.21.15.112:0/3354809516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 msgr2=0x7f30d414b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d259c640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 0x7f30d414b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d259c640 1 -- 172.21.15.112:0/3354809516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d4145d20 msgr2=0x7f30d41e5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d259c640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d4145d20 0x7f30d41e5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d259c640 1 -- 172.21.15.112:0/3354809516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d414a1e0 con 0x7f30d40baed0 2024-04-25T00:55:45.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f0783fff640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f077c038390 0x7f077c0ee060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1baffd640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 0x7fa1bc0ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d359e640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d4145d20 0x7f30d41e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb50865f640 1 -- 172.21.15.112:0/38249275 shutdown_connections 2024-04-25T00:55:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb50865f640 1 --2- 172.21.15.112:0/38249275 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50014f500 0x7fb50014f950 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb50865f640 1 --2- 172.21.15.112:0/38249275 >> v2:172.21.15.112:3300/0 conn(0x7fb5000af900 0x7fb50014efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fb50865f640 1 -- 172.21.15.112:0/38249275 >> 172.21.15.112:0/38249275 conn(0x7fb5000b5980 msgr2=0x7fb5000b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1bbfff640 1 -- 172.21.15.112:0/1074301067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc06c7c0 msgr2=0x7fa1bc143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1bbfff640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc06c7c0 0x7fa1bc143370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1bbfff640 1 -- 172.21.15.112:0/1074301067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 msgr2=0x7fa1bc0ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fa1bbfff640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 0x7fa1bc0ab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7f30d259c640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 0x7f30d41e4cd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00bd70 tx=0x7f30cc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f078b073640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f077c037470 con 0x7f077c0eb6c0 2024-04-25T00:55:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f078b073640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f077c0f7800 con 0x7f077c038390 2024-04-25T00:55:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f078b073640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f077c0ec090 con 0x7f077c035fc0 2024-04-25T00:55:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f0789610640 1 -- 172.21.15.112:0/2633454969 learned_addr learned my addr 172.21.15.112:0/2633454969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.525 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fa1bbfff640 1 -- 172.21.15.112:0/1074301067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1bc0aa420 con 0x7fa1bc0b5500 2024-04-25T00:55:45.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 -- 172.21.15.112:0/38249275 shutdown_connections 2024-04-25T00:55:45.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fa1bbfff640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1bc0b5500 0x7fa1bc0ab5f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400efc0 tx=0x7fa1b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9dbb3ea640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40adfd0 0x7f9db414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9dbb3ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40ae3b0 0x7f9db4071010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9dbb3ea640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9db40b2780 0x7f9db413d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9dbb3ea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41508a0 con 0x7f9db40ae3b0 2024-04-25T00:55:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9dbb3ea640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4150720 con 0x7f9db40b2780 2024-04-25T00:55:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9dbb3ea640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9db4150a20 con 0x7f9db40adfd0 2024-04-25T00:55:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cdad3c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4cd40b4a70 0x7f4cd40b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fa1a3fff640 1 -- 172.21.15.112:0/1074301067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4018020 con 0x7fa1bc0b5500 2024-04-25T00:55:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40ae3b0 0x7f9db4071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40ae3b0 0x7f9db4071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51296/0 (socket says 172.21.15.112:51296) 2024-04-25T00:55:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 -- 172.21.15.112:0/2276139250 learned_addr learned my addr 172.21.15.112:0/2276139250 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fa1a3fff640 1 -- 172.21.15.112:0/1074301067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1b4014480 con 0x7fa1bc0b5500 2024-04-25T00:55:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fa1a3fff640 1 -- 172.21.15.112:0/1074301067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4014870 con 0x7fa1bc0b5500 2024-04-25T00:55:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f0788e0f640 1 -- 172.21.15.112:0/2633454969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f077c038390 msgr2=0x7f077c0ee060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f0788e0f640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f077c038390 0x7f077c0ee060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f0788e0f640 1 -- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0eb6c0 msgr2=0x7f077c0ebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f0788e0f640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0eb6c0 0x7f077c0ebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f0788e0f640 1 -- 172.21.15.112:0/2633454969 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c0e73e0 con 0x7f077c035fc0 2024-04-25T00:55:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f0788e0f640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c035fc0 0x7f077c0edb20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f077000c6b0 tx=0x7f077000cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f07817fa640 1 -- 172.21.15.112:0/2633454969 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770007960 con 0x7f077c035fc0 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f078b073640 1 -- 172.21.15.112:0/2633454969 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c0e76c0 con 0x7f077c035fc0 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f078b073640 1 -- 172.21.15.112:0/2633454969 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c0e2a70 con 0x7f077c035fc0 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.492+0000 7fbfecd5f640 1 -- 172.21.15.112:0/507886368 wait complete. 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfecd5f640 1 Processor -- start 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfecd5f640 1 -- start start 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfecd5f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe80b0590 0x7fbfe806f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfecd5f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe814b650 0x7fbfe81f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfecd5f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe814f870 0x7fbfe8071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfecd5f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe80b57f0 con 0x7fbfe814f870 2024-04-25T00:55:45.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfecd5f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe80b5670 con 0x7fbfe80b0590 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfecd5f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe80b4c80 con 0x7fbfe814b650 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe80b0590 0x7fbfe806f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe77fe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe80b0590 0x7fbfe806f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41014/0 (socket says 172.21.15.112:41014) 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe77fe640 1 -- 172.21.15.112:0/424070655 learned_addr learned my addr 172.21.15.112:0/424070655 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe6ffd640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe814b650 0x7fbfe81f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cdad3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40af0e0 0x7f4cd4146d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cdad3c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4cd4147260 0x7f4cd406c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cdad3c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd40aec50 con 0x7f4cd40b4a70 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cdad3c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd40b0400 con 0x7f4cd4147260 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cdad3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd40b4e50 con 0x7f4cd40af0e0 2024-04-25T00:55:45.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd92d9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4cd4147260 0x7f4cd406c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd3fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40af0e0 0x7f4cd4146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd92d9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4cd4147260 0x7f4cd406c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41002/0 (socket says 172.21.15.112:41002) 2024-04-25T00:55:45.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd92d9640 1 -- 172.21.15.112:0/2609724501 learned_addr learned my addr 172.21.15.112:0/2609724501 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fa1c0a46640 1 -- 172.21.15.112:0/1074301067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1bc144010 con 0x7fa1bc0b5500 2024-04-25T00:55:45.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fa1c0a46640 1 -- 172.21.15.112:0/1074301067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1bc140bb0 con 0x7fa1bc0b5500 2024-04-25T00:55:45.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 -- 172.21.15.112:0/38249275 wait complete. 2024-04-25T00:55:45.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 Processor -- start 2024-04-25T00:55:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db9987640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9db40b2780 0x7f9db413d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db9186640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40adfd0 0x7f9db414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 -- 172.21.15.112:0/2276139250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40adfd0 msgr2=0x7f9db414d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40adfd0 0x7f9db414d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 -- 172.21.15.112:0/2276139250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9db40b2780 msgr2=0x7f9db413d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9db40b2780 0x7f9db413d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 -- 172.21.15.112:0/2276139250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db40716f0 con 0x7f9db40ae3b0 2024-04-25T00:55:45.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f9db8985640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40ae3b0 0x7f9db4071010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9da400c7f0 tx=0x7f9da400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f07817fa640 1 -- 172.21.15.112:0/2633454969 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0770008040 con 0x7f077c035fc0 2024-04-25T00:55:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f078b073640 1 -- 172.21.15.112:0/2633454969 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077c0eb390 con 0x7f077c035fc0 2024-04-25T00:55:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f07817fa640 1 -- 172.21.15.112:0/2633454969 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770007d40 con 0x7f077c035fc0 2024-04-25T00:55:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe6ffd640 1 -- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe80b0590 msgr2=0x7fbfe806f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe6ffd640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe80b0590 0x7fbfe806f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe6ffd640 1 -- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe814f870 msgr2=0x7fbfe8071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe6ffd640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe814f870 0x7fbfe8071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe6ffd640 1 -- 172.21.15.112:0/424070655 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe81efe20 con 0x7fbfe814b650 2024-04-25T00:55:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe77fe640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe80b0590 0x7fbfe806f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfe6ffd640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe814b650 0x7fbfe81f08c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c7c0 tx=0x7fbfd800cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 -- start start 2024-04-25T00:55:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50006c520 0x7fb5000715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5000af900 0x7fb5000b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb50014f500 0x7fb5000b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb500156de0 con 0x7fb5000af900 2024-04-25T00:55:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb5000ae140 con 0x7fb50014f500 2024-04-25T00:55:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb50865f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb500156f60 con 0x7fb50006c520 2024-04-25T00:55:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fbfcffff640 1 -- 172.21.15.112:0/424070655 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8015070 con 0x7fbfe814b650 2024-04-25T00:55:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd3fff640 1 -- 172.21.15.112:0/2609724501 >> v2:172.21.15.187:3300/0 conn(0x7f4cd4147260 msgr2=0x7f4cd406c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd3fff640 1 --2- 172.21.15.112:0/2609724501 >> v2:172.21.15.187:3300/0 conn(0x7f4cd4147260 0x7f4cd406c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd3fff640 1 -- 172.21.15.112:0/2609724501 >> v2:172.21.15.112:3300/0 conn(0x7f4cd40b4a70 msgr2=0x7f4cd40b0b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:55:45.536 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd3fff640 1 --2- 172.21.15.112:0/2609724501 >> v2:172.21.15.112:3300/0 conn(0x7f4cd40b4a70 0x7f4cd40b0b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7fb506bfc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb50014f500 0x7fb5000b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.496+0000 7f4cd3fff640 1 -- 172.21.15.112:0/2609724501 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd40b06e0 con 0x7f4cd40af0e0 2024-04-25T00:55:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb506bfc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb50014f500 0x7fb5000b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41018/0 (socket says 172.21.15.112:41018) 2024-04-25T00:55:45.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb506bfc640 1 -- 172.21.15.112:0/4256473264 learned_addr learned my addr 172.21.15.112:0/4256473264 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fa1a3fff640 1 -- 172.21.15.112:0/1074301067 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa1b401b020 con 0x7fa1bc0b5500 2024-04-25T00:55:45.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f9da9ffb640 1 -- 172.21.15.112:0/2276139250 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4015070 con 0x7f9db40ae3b0 2024-04-25T00:55:45.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb505bfa640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5000af900 0x7fb5000b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb5063fb640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50006c520 0x7fb5000715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f30bb7fe640 1 -- 172.21.15.112:0/3354809516 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc019070 con 0x7f30d40baed0 2024-04-25T00:55:45.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfcffff640 1 -- 172.21.15.112:0/424070655 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfd80079b0 con 0x7fbfe814b650 2024-04-25T00:55:45.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfcffff640 1 -- 172.21.15.112:0/424070655 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8007d60 con 0x7fbfe814b650 2024-04-25T00:55:45.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe8071130 con 0x7fbfe814b650 2024-04-25T00:55:45.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe81f00c0 con 0x7fbfe814b650 2024-04-25T00:55:45.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f4cd3fff640 1 --2- 172.21.15.112:0/2609724501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40af0e0 0x7f4cd4146d20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4072cc0 tx=0x7f4cc8020790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6f4ff16215cb6ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f9da9ffb640 1 -- 172.21.15.112:0/2276139250 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da4004240 con 0x7f9db40ae3b0 2024-04-25T00:55:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f9da9ffb640 1 -- 172.21.15.112:0/2276139250 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4003620 con 0x7f9db40ae3b0 2024-04-25T00:55:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2276139250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db41e8040 con 0x7f9db40ae3b0 2024-04-25T00:55:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2276139250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db41e7d30 con 0x7f9db40ae3b0 2024-04-25T00:55:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d41e47b0 con 0x7f30d40baed0 2024-04-25T00:55:45.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41496a0 con 0x7f30d40baed0 2024-04-25T00:55:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d404f580 con 0x7f30d40baed0 2024-04-25T00:55:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb505bfa640 1 -- 172.21.15.112:0/4256473264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50006c520 msgr2=0x7fb5000715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb505bfa640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50006c520 0x7fb5000715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb505bfa640 1 -- 172.21.15.112:0/4256473264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb50014f500 msgr2=0x7fb5000b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb505bfa640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb50014f500 0x7fb5000b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb505bfa640 1 -- 172.21.15.112:0/4256473264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb50013e460 con 0x7fb5000af900 2024-04-25T00:55:45.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb5063fb640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50006c520 0x7fb5000715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fa1a3fff640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1a80678f0 0x7fa1a8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fa1a3fff640 1 -- 172.21.15.112:0/1074301067 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa1b4086750 con 0x7fa1bc0b5500 2024-04-25T00:55:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fa1bb7fe640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1a80678f0 0x7fa1a8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fa1c0a46640 1 -- 172.21.15.112:0/1074301067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1bc04f510 con 0x7fa1bc0b5500 2024-04-25T00:55:45.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fb505bfa640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5000af900 0x7fb5000b8280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800c7f0 tx=0x7fb4f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2276139250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db41e8750 con 0x7f9db40ae3b0 2024-04-25T00:55:45.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe804f510 con 0x7fbfe814b650 2024-04-25T00:55:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f07817fa640 1 -- 172.21.15.112:0/2633454969 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f077000f410 con 0x7f077c035fc0 2024-04-25T00:55:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f07817fa640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07680678c0 0x7f0768069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f07817fa640 1 -- 172.21.15.112:0/2633454969 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0770086480 con 0x7f077c035fc0 2024-04-25T00:55:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f0783fff640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07680678c0 0x7f0768069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfcffff640 1 -- 172.21.15.112:0/424070655 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbfd8018070 con 0x7fbfe814b650 2024-04-25T00:55:45.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfcffff640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbfd0067a20 0x7fbfd0069ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7f0783fff640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07680678c0 0x7f0768069d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07780b1000 tx=0x7f077800c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfcffff640 1 -- 172.21.15.112:0/424070655 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbfd8085a20 con 0x7fbfe814b650 2024-04-25T00:55:45.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fbfe77fe640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbfd0067a20 0x7fbfd0069ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7fbfe77fe640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbfd0067a20 0x7fbfd0069ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc01c560 tx=0x7fbfdc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.500+0000 7fa1bb7fe640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1a80678f0 0x7fa1a8069db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac000f70 tx=0x7fa1ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f30bb7fe640 1 -- 172.21.15.112:0/3354809516 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30cc01e440 con 0x7f30d40baed0 2024-04-25T00:55:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f30bb7fe640 1 -- 172.21.15.112:0/3354809516 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc023060 con 0x7f30d40baed0 2024-04-25T00:55:45.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7fb4feffd640 1 -- 172.21.15.112:0/4256473264 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8015070 con 0x7fb5000af900 2024-04-25T00:55:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f9da9ffb640 1 -- 172.21.15.112:0/2276139250 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9da40039d0 con 0x7f9db40ae3b0 2024-04-25T00:55:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f9da9ffb640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9d980678c0 0x7f9d98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f9db9186640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9d980678c0 0x7f9d98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f9db9186640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9d980678c0 0x7f9d98069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9db0001650 tx=0x7f9db0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f30bb7fe640 1 -- 172.21.15.112:0/3354809516 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f30cc01c070 con 0x7f30d40baed0 2024-04-25T00:55:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f30bb7fe640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30c00678c0 0x7f30c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f30d2d9d640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30c00678c0 0x7f30c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb50014c760 con 0x7fb5000af900 2024-04-25T00:55:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb500071f90 con 0x7fb5000af900 2024-04-25T00:55:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50004f580 con 0x7fb5000af900 2024-04-25T00:55:45.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.504+0000 7f30d2d9d640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30c00678c0 0x7f30c0069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f30c40af000 tx=0x7f30c40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d87a7640 1 Processor -- start 2024-04-25T00:55:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d87a7640 1 -- start start 2024-04-25T00:55:45.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d87a7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d87a7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47d014d940 0x7f47d014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d87a7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00b5020 con 0x7f47d0072340 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d87a7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f47d00b5900 con 0x7f47d014d940 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d87a7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47d0072720 con 0x7f47d013efd0 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d6543640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f47d0072340 0x7f47d01485c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57296/0 (socket says 172.21.15.112:57296) 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d6543640 1 -- 172.21.15.112:0/2364532320 learned_addr learned my addr 172.21.15.112:0/2364532320 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d6d44640 1 --2- 172.21.15.112:0/2364532320 >> v2:172.21.15.112:3300/0 conn(0x7f47d014d940 0x7f47d014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d5d42640 1 --2- 172.21.15.112:0/2364532320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d6d44640 1 -- 172.21.15.112:0/2364532320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 msgr2=0x7f47d013f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d6d44640 1 --2- 172.21.15.112:0/2364532320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d6d44640 1 -- 172.21.15.112:0/2364532320 >> v1:172.21.15.112:6789/0 conn(0x7f47d0072340 legacy=0x7f47d01485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d6d44640 1 -- 172.21.15.112:0/2364532320 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00b5620 con 0x7f47d014d940 2024-04-25T00:55:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7fa1a3fff640 1 -- 172.21.15.112:0/1074301067 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa1b4022070 con 0x7fa1bc0b5500 2024-04-25T00:55:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7fb4feffd640 1 -- 172.21.15.112:0/4256473264 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f8004430 con 0x7fb5000af900 2024-04-25T00:55:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7fb4feffd640 1 -- 172.21.15.112:0/4256473264 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f801e060 con 0x7fb5000af900 2024-04-25T00:55:45.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f9da9ffb640 1 -- 172.21.15.112:0/2276139250 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9da40546d0 con 0x7f9db40ae3b0 2024-04-25T00:55:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f30bb7fe640 1 -- 172.21.15.112:0/3354809516 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f30cc054670 con 0x7f30d40baed0 2024-04-25T00:55:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.508+0000 7f47d6d44640 1 --2- 172.21.15.112:0/2364532320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d014d940 0x7f47d014fd30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009dc0 tx=0x7f47c801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c88aa6d6eb91423 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.512+0000 7f9da9ffb640 1 -- 172.21.15.112:0/2276139250 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9da4056d30 con 0x7f9db40ae3b0 2024-04-25T00:55:45.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.512+0000 7f4cd37fe640 1 -- 172.21.15.112:0/2609724501 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80ad400 con 0x7f4cd40af0e0 2024-04-25T00:55:45.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.516+0000 7f07817fa640 1 -- 172.21.15.112:0/2633454969 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0770026020 con 0x7f077c035fc0 2024-04-25T00:55:45.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.516+0000 7f47d5541640 1 -- 172.21.15.112:0/2364532320 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c80b2070 con 0x7f47d014d940 2024-04-25T00:55:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.516+0000 7fb4feffd640 1 -- 172.21.15.112:0/4256473264 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb4f8007820 con 0x7fb5000af900 2024-04-25T00:55:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.516+0000 7fb4feffd640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb4e80678f0 0x7fb4e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.516+0000 7fb5063fb640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb4e80678f0 0x7fb4e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.516+0000 7fb5063fb640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb4e80678f0 0x7fb4e8069db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0000f70 tx=0x7fb4f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.520+0000 7f30bb7fe640 1 -- 172.21.15.112:0/3354809516 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f30cc0565d0 con 0x7f30d40baed0 2024-04-25T00:55:45.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.520+0000 7fb4feffd640 1 -- 172.21.15.112:0/4256473264 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb4f80545d0 con 0x7fb5000af900 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.520+0000 7f47d5541640 1 -- 172.21.15.112:0/2364532320 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c80a5900 con 0x7f47d014d940 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.520+0000 7f47d5541640 1 -- 172.21.15.112:0/2364532320 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c80a5cd0 con 0x7f47d014d940 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.520+0000 7f47d87a7640 1 -- 172.21.15.112:0/2364532320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d014d940 msgr2=0x7f47d014fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.520+0000 7f47d87a7640 1 --2- 172.21.15.112:0/2364532320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d014d940 0x7f47d014fd30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009dc0 tx=0x7f47c801c950 comp rx=0 tx=0).stop 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 -- 172.21.15.112:0/2364532320 shutdown_connections 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 --2- 172.21.15.112:0/2364532320 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d014d940 0x7f47d014fd30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 --2- 172.21.15.112:0/2364532320 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 -- 172.21.15.112:0/2364532320 >> 172.21.15.112:0/2364532320 conn(0x7f47d006b210 msgr2=0x7f47d006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 -- 172.21.15.112:0/2364532320 shutdown_connections 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 -- 172.21.15.112:0/2364532320 wait complete. 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 Processor -- start 2024-04-25T00:55:45.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 -- start start 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d0072340 0x7f47d014efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d6543640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d0072340 0x7f47d014efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d6543640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d0072340 0x7f47d014efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51336/0 (socket says 172.21.15.112:51336) 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d6543640 1 -- 172.21.15.112:0/488237806 learned_addr learned my addr 172.21.15.112:0/488237806 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.524+0000 7f47d87a7640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d87a7640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47d014d940 0x7f47d00b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d87a7640 1 -- 172.21.15.112:0/488237806 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00b3ac0 con 0x7f47d0072340 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d87a7640 1 -- 172.21.15.112:0/488237806 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00b52b0 con 0x7f47d014d940 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d5d42640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d87a7640 1 -- 172.21.15.112:0/488237806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f47d00b3c40 con 0x7f47d013efd0 2024-04-25T00:55:45.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d6d44640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47d014d940 0x7f47d00b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7fbfcffff640 1 -- 172.21.15.112:0/424070655 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbfd8055fa0 con 0x7fbfe814b650 2024-04-25T00:55:45.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d5d42640 1 -- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47d014d940 msgr2=0x7f47d00b88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d5d42640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47d014d940 0x7f47d00b88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d5d42640 1 -- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d0072340 msgr2=0x7f47d014efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cd37fe640 1 -- 172.21.15.112:0/2609724501 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc80ad5a0 con 0x7f4cd40af0e0 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cd37fe640 1 -- 172.21.15.112:0/2609724501 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80ad970 con 0x7f4cd40af0e0 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 -- 172.21.15.112:0/2609724501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40af0e0 msgr2=0x7f4cd4146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/2609724501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40af0e0 0x7f4cd4146d20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4072cc0 tx=0x7f4cc8020790 comp rx=0 tx=0).stop 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 -- 172.21.15.112:0/2609724501 shutdown_connections 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/2609724501 >> v2:172.21.15.187:3300/0 conn(0x7f4cd4147260 0x7f4cd406c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/2609724501 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40af0e0 0x7f4cd4146d20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/2609724501 >> v2:172.21.15.112:3300/0 conn(0x7f4cd40b4a70 0x7f4cd40b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d5d42640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d0072340 0x7f47d014efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d5d42640 1 -- 172.21.15.112:0/488237806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d01402e0 con 0x7f47d013efd0 2024-04-25T00:55:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d6543640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d0072340 0x7f47d014efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 -- 172.21.15.112:0/2609724501 >> 172.21.15.112:0/2609724501 conn(0x7f4cd40b8ac0 msgr2=0x7f4cd40b8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 -- 172.21.15.112:0/2609724501 shutdown_connections 2024-04-25T00:55:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 -- 172.21.15.112:0/2609724501 wait complete. 2024-04-25T00:55:45.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d5d42640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013fbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f47c000d5a0 tx=0x7f47c000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7fb4feffd640 1 -- 172.21.15.112:0/4256473264 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb4f8056530 con 0x7fb5000af900 2024-04-25T00:55:45.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47c6ffd640 1 -- 172.21.15.112:0/488237806 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0019070 con 0x7f47d013efd0 2024-04-25T00:55:45.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47c6ffd640 1 -- 172.21.15.112:0/488237806 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c0004240 con 0x7f47d013efd0 2024-04-25T00:55:45.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 Processor -- start 2024-04-25T00:55:45.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f4cdad3c640 1 -- start start 2024-04-25T00:55:45.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d87a7640 1 -- 172.21.15.112:0/488237806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d00b9d10 con 0x7f47d013efd0 2024-04-25T00:55:45.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.528+0000 7f47d87a7640 1 -- 172.21.15.112:0/488237806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d00ba2d0 con 0x7f47d013efd0 2024-04-25T00:55:45.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cdad3c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd40af0e0 0x7f4cd4145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cdad3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40b4a70 0x7f4cd41df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cdad3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cd4147260 0x7f4cd41dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cdad3c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd40ae230 con 0x7f4cd40af0e0 2024-04-25T00:55:45.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cdad3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd40af5b0 con 0x7f4cd4147260 2024-04-25T00:55:45.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cdad3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd40ae3b0 con 0x7f4cd40b4a70 2024-04-25T00:55:45.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f47c6ffd640 1 -- 172.21.15.112:0/488237806 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c0003620 con 0x7f47d013efd0 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd40af0e0 0x7f4cd4145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd92d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cd4147260 0x7f4cd41dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd40af0e0 0x7f4cd4145510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51350/0 (socket says 172.21.15.112:51350) 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 -- 172.21.15.112:0/1867058944 learned_addr learned my addr 172.21.15.112:0/1867058944 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 -- 172.21.15.112:0/1867058944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40b4a70 msgr2=0x7f4cd41df7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40b4a70 0x7f4cd41df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 -- 172.21.15.112:0/1867058944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cd4147260 msgr2=0x7f4cd41dfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cd4147260 0x7f4cd41dfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 -- 172.21.15.112:0/1867058944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd41e0320 con 0x7f4cd40af0e0 2024-04-25T00:55:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f47c4ff9640 1 -- 172.21.15.112:0/488237806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d00b9b10 con 0x7f47d013efd0 2024-04-25T00:55:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f47c6ffd640 1 -- 172.21.15.112:0/488237806 <== mon.2 v2:172.21.15.187:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47c001c070 con 0x7f47d013efd0 2024-04-25T00:55:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd8ad8640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd40af0e0 0x7f4cd4145510 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b4d0 tx=0x7f4cc000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cd17fa640 1 -- 172.21.15.112:0/1867058944 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0014070 con 0x7f4cd40af0e0 2024-04-25T00:55:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f47c6ffd640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f47b4067910 0x7f47b4069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd41e0600 con 0x7f4cd40af0e0 2024-04-25T00:55:45.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.532+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd409aa30 con 0x7f4cd40af0e0 2024-04-25T00:55:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.536+0000 7f47c6ffd640 1 -- 172.21.15.112:0/488237806 <== mon.2 v2:172.21.15.187:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f47c0085420 con 0x7f47d013efd0 2024-04-25T00:55:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.536+0000 7f47d6543640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f47b4067910 0x7f47b4069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.536+0000 7f4cd17fa640 1 -- 172.21.15.112:0/1867058944 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc0010870 con 0x7f4cd40af0e0 2024-04-25T00:55:45.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.536+0000 7f47d6543640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f47b4067910 0x7f47b4069dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47cc007fd0 tx=0x7f47cc007f20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.536+0000 7f4cd17fa640 1 -- 172.21.15.112:0/1867058944 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc001e060 con 0x7f4cd40af0e0 2024-04-25T00:55:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.536+0000 7f4cd17fa640 1 -- 172.21.15.112:0/1867058944 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4cc0017070 con 0x7f4cd40af0e0 2024-04-25T00:55:45.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.540+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cb40052a0 con 0x7f4cd40af0e0 2024-04-25T00:55:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.540+0000 7f4cd17fa640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4cb80678c0 0x7f4cb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.540+0000 7f4cd3fff640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4cb80678c0 0x7f4cb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.540+0000 7f4cd3fff640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4cb80678c0 0x7f4cb8069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80b2000 tx=0x7f4cc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.544+0000 7f4cd17fa640 1 -- 172.21.15.112:0/1867058944 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4cc0084170 con 0x7f4cd40af0e0 2024-04-25T00:55:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.548+0000 7f47c6ffd640 1 -- 172.21.15.112:0/488237806 <== mon.2 v2:172.21.15.187:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f47c0055980 con 0x7f47d013efd0 2024-04-25T00:55:45.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.552+0000 7f4cd17fa640 1 -- 172.21.15.112:0/1867058944 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4cc00565f0 con 0x7f4cd40af0e0 2024-04-25T00:55:45.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.812+0000 7f078b073640 1 -- 172.21.15.112:0/2633454969 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f077c0053a0 con 0x7f077c035fc0 2024-04-25T00:55:45.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.816+0000 7f07817fa640 1 -- 172.21.15.112:0/2633454969 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0770014030 con 0x7f077c035fc0 2024-04-25T00:55:45.822 INFO:teuthology.orchestra.run.smithi112.stdout:55834574856 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.816+0000 7f076effd640 1 -- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07680678c0 msgr2=0x7f0768069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.816+0000 7f076effd640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07680678c0 0x7f0768069d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07780b1000 tx=0x7f077800c040 comp rx=0 tx=0).stop 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.816+0000 7f076effd640 1 -- 172.21.15.112:0/2633454969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c035fc0 msgr2=0x7f077c0edb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.816+0000 7f076effd640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c035fc0 0x7f077c0edb20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f077000c6b0 tx=0x7f077000cb70 comp rx=0 tx=0).stop 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.816+0000 7f076effd640 1 -- 172.21.15.112:0/2633454969 shutdown_connections 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.816+0000 7f076effd640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f077c0eb6c0 0x7f077c0ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.816+0000 7f076effd640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07680678c0 0x7f0768069d80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.820+0000 7f076effd640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f077c038390 0x7f077c0ee060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.820+0000 7f076effd640 1 --2- 172.21.15.112:0/2633454969 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f077c035fc0 0x7f077c0edb20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.820+0000 7f076effd640 1 -- 172.21.15.112:0/2633454969 >> 172.21.15.112:0/2633454969 conn(0x7f077c03a860 msgr2=0x7f077c0055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.820+0000 7f076effd640 1 -- 172.21.15.112:0/2633454969 shutdown_connections 2024-04-25T00:55:45.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.820+0000 7f076effd640 1 -- 172.21.15.112:0/2633454969 wait complete. 2024-04-25T00:55:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.824+0000 7fa1c0a46640 1 -- 172.21.15.112:0/1074301067 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa1bc0b1650 con 0x7fa1bc0b5500 2024-04-25T00:55:45.829 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.824+0000 7fa1a3fff640 1 -- 172.21.15.112:0/1074301067 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1b4010040 con 0x7fa1bc0b5500 2024-04-25T00:55:45.830 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 -- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1a80678f0 msgr2=0x7fa1a8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1a80678f0 0x7fa1a8069db0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac000f70 tx=0x7fa1ac009040 comp rx=0 tx=0).stop 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 -- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1bc0b5500 msgr2=0x7fa1bc0ab5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1bc0b5500 0x7fa1bc0ab5f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400efc0 tx=0x7fa1b400c490 comp rx=0 tx=0).stop 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 -- 172.21.15.112:0/1074301067 shutdown_connections 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1bc0b5500 0x7fa1bc0ab5f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1a80678f0 0x7fa1a8069db0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1bc0afd70 0x7fa1bc0ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 --2- 172.21.15.112:0/1074301067 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa1bc06c7c0 0x7fa1bc143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 -- 172.21.15.112:0/1074301067 >> 172.21.15.112:0/1074301067 conn(0x7fa1bc06bca0 msgr2=0x7fa1bc06c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 -- 172.21.15.112:0/1074301067 shutdown_connections 2024-04-25T00:55:45.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.832+0000 7fa1a27fc640 1 -- 172.21.15.112:0/1074301067 wait complete. 2024-04-25T00:55:45.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.840+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f30d4072eb0 con 0x7f30d40baed0 2024-04-25T00:55:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.840+0000 7f9dbb3ea640 1 -- 172.21.15.112:0/2276139250 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9db404f580 con 0x7f9db40ae3b0 2024-04-25T00:55:45.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.840+0000 7f30bb7fe640 1 -- 172.21.15.112:0/3354809516 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f30cc05a8d0 con 0x7f30d40baed0 2024-04-25T00:55:45.846 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.840+0000 7f9da9ffb640 1 -- 172.21.15.112:0/2276139250 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9da4056630 con 0x7f9db40ae3b0 2024-04-25T00:55:45.847 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2024-04-25T00:55:45.848 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30c00678c0 msgr2=0x7f30c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30c00678c0 0x7f30c0069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f30c40af000 tx=0x7f30c40a7000 comp rx=0 tx=0).stop 2024-04-25T00:55:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 -- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9d980678c0 msgr2=0x7f9d98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9d980678c0 0x7f9d98069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9db0001650 tx=0x7f9db0009040 comp rx=0 tx=0).stop 2024-04-25T00:55:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 -- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40ae3b0 msgr2=0x7f9db4071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40ae3b0 0x7f9db4071010 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9da400c7f0 tx=0x7f9da400ccb0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 msgr2=0x7f30d41e4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 0x7f30d41e4cd0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00bd70 tx=0x7f30cc00be60 comp rx=0 tx=0).stop 2024-04-25T00:55:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 -- 172.21.15.112:0/2276139250 shutdown_connections 2024-04-25T00:55:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9d980678c0 0x7f9d98069d80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9db40b2780 0x7f9db413d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9db40ae3b0 0x7f9db4071010 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 --2- 172.21.15.112:0/2276139250 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9db40adfd0 0x7f9db414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 -- 172.21.15.112:0/2276139250 >> 172.21.15.112:0/2276139250 conn(0x7f9db40b5980 msgr2=0x7f9db413fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 shutdown_connections 2024-04-25T00:55:45.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f30c00678c0 0x7f30c0069d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f30d4145d20 0x7f30d41e5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30d40baed0 0x7f30d41e4cd0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 --2- 172.21.15.112:0/3354809516 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f30d40ba2d0 0x7f30d414b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 >> 172.21.15.112:0/3354809516 conn(0x7f30d4067120 msgr2=0x7f30d40b6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 -- 172.21.15.112:0/2276139250 shutdown_connections 2024-04-25T00:55:45.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f9d977fe640 1 -- 172.21.15.112:0/2276139250 wait complete. 2024-04-25T00:55:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 shutdown_connections 2024-04-25T00:55:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.844+0000 7f30d907e640 1 -- 172.21.15.112:0/3354809516 wait complete. 2024-04-25T00:55:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.848+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fbfe80b7d20 con 0x7fbfe814b650 2024-04-25T00:55:45.852 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.848+0000 7fbfcffff640 1 -- 172.21.15.112:0/424070655 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbfd805a2a0 con 0x7fbfe814b650 2024-04-25T00:55:45.853 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.848+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbfd0067a20 msgr2=0x7fbfd0069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.848+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbfd0067a20 0x7fbfd0069ee0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc01c560 tx=0x7fbfdc0a7000 comp rx=0 tx=0).stop 2024-04-25T00:55:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.848+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe814b650 msgr2=0x7fbfe81f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.848+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe814b650 0x7fbfe81f08c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800c7c0 tx=0x7fbfd800cc80 comp rx=0 tx=0).stop 2024-04-25T00:55:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.852+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 shutdown_connections 2024-04-25T00:55:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.852+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbfe814f870 0x7fbfe8071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.852+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbfe814b650 0x7fbfe81f08c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.852+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbfd0067a20 0x7fbfd0069ee0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.852+0000 7fbfecd5f640 1 --2- 172.21.15.112:0/424070655 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbfe80b0590 0x7fbfe806f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.852+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 >> 172.21.15.112:0/424070655 conn(0x7fbfe806bca0 msgr2=0x7fbfe80b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.852+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 shutdown_connections 2024-04-25T00:55:45.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.852+0000 7fbfecd5f640 1 -- 172.21.15.112:0/424070655 wait complete. 2024-04-25T00:55:45.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-04-25T00:55:45.860 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:45.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.860+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb5000b0f50 con 0x7fb5000af900 2024-04-25T00:55:45.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.864+0000 7fb4feffd640 1 -- 172.21.15.112:0/4256473264 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb4f805a830 con 0x7fb5000af900 2024-04-25T00:55:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.864+0000 7f47c4ff9640 1 -- 172.21.15.112:0/488237806 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f47d01dced0 con 0x7f47d013efd0 2024-04-25T00:55:45.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.864+0000 7f47c6ffd640 1 -- 172.21.15.112:0/488237806 <== mon.2 v2:172.21.15.187:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47c0059c80 con 0x7f47d013efd0 2024-04-25T00:55:45.869 INFO:teuthology.orchestra.run.smithi112.stdout:55834574853 2024-04-25T00:55:45.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 -- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f47b4067910 msgr2=0x7f47b4069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f47b4067910 0x7f47b4069dd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47cc007fd0 tx=0x7f47cc007f20 comp rx=0 tx=0).stop 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 -- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 msgr2=0x7f47d013fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013fbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f47c000d5a0 tx=0x7f47c000da60 comp rx=0 tx=0).stop 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 -- 172.21.15.112:0/488237806 shutdown_connections 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f47b4067910 0x7f47b4069dd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f47d014d940 0x7f47d00b88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f47d013efd0 0x7f47d013fbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 --2- 172.21.15.112:0/488237806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d0072340 0x7f47d014efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 -- 172.21.15.112:0/488237806 >> 172.21.15.112:0/488237806 conn(0x7f47d006b210 msgr2=0x7f47d00676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 -- 172.21.15.112:0/488237806 shutdown_connections 2024-04-25T00:55:45.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.868+0000 7f47c4ff9640 1 -- 172.21.15.112:0/488237806 wait complete. 2024-04-25T00:55:45.875 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-04-25T00:55:45.876 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:45.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb4e80678f0 msgr2=0x7fb4e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb4e80678f0 0x7fb4e8069db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0000f70 tx=0x7fb4f0009040 comp rx=0 tx=0).stop 2024-04-25T00:55:45.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5000af900 msgr2=0x7fb5000b8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5000af900 0x7fb5000b8280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800c7f0 tx=0x7fb4f800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 shutdown_connections 2024-04-25T00:55:45.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb4e80678f0 0x7fb4e8069db0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb50014f500 0x7fb5000b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb5000af900 0x7fb5000b8280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 --2- 172.21.15.112:0/4256473264 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb50006c520 0x7fb5000715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 >> 172.21.15.112:0/4256473264 conn(0x7fb5000b5980 msgr2=0x7fb50013bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 shutdown_connections 2024-04-25T00:55:45.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.872+0000 7fb50865f640 1 -- 172.21.15.112:0/4256473264 wait complete. 2024-04-25T00:55:45.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-04-25T00:55:45.879 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:45.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-25T00:55:45.880 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:45.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4cb4005590 con 0x7f4cd40af0e0 2024-04-25T00:55:45.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cd17fa640 1 -- 172.21.15.112:0/1867058944 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4cc0055ef0 con 0x7f4cd40af0e0 2024-04-25T00:55:45.883 INFO:teuthology.orchestra.run.smithi112.stdout:55834574855 2024-04-25T00:55:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4cb80678c0 msgr2=0x7f4cb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4cb80678c0 0x7f4cb8069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80b2000 tx=0x7f4cc8008040 comp rx=0 tx=0).stop 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd40af0e0 msgr2=0x7f4cd4145510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd40af0e0 0x7f4cd4145510 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000b4d0 tx=0x7f4cc000b990 comp rx=0 tx=0).stop 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 shutdown_connections 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4cb80678c0 0x7f4cb8069d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4cd4147260 0x7f4cd41dfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4cd40b4a70 0x7f4cd41df7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 --2- 172.21.15.112:0/1867058944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4cd40af0e0 0x7f4cd4145510 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 >> 172.21.15.112:0/1867058944 conn(0x7f4cd40b8ac0 msgr2=0x7f4cd4143800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 shutdown_connections 2024-04-25T00:55:45.886 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:45.880+0000 7f4cdad3c640 1 -- 172.21.15.112:0/1867058944 wait complete. 2024-04-25T00:55:45.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-04-25T00:55:45.900 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:45.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-04-25T00:55:45.901 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:45.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2024-04-25T00:55:45.903 DEBUG:teuthology.parallel:result is None 2024-04-25T00:55:45.903 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:55:45.904 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:55:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.040+0000 7efc37bac640 1 Processor -- start 2024-04-25T00:55:46.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.040+0000 7efc37bac640 1 -- start start 2024-04-25T00:55:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc37bac640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7efc30074590 0x7efc300b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc37bac640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc300b3a80 0x7efc301422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc37bac640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc3009bec0 con 0x7efc301427e0 2024-04-25T00:55:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc37bac640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7efc3009c6e0 con 0x7efc300b3a80 2024-04-25T00:55:46.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc37bac640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7efc30074970 con 0x7efc30074590 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35147640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc300b3a80 0x7efc301422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35147640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efc300b3a80 0x7efc301422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51354/0 (socket says 172.21.15.112:51354) 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc36149640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7efc301427e0 0x7efc3006c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57302/0 (socket says 172.21.15.112:57302) 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35147640 1 -- 172.21.15.112:0/2215956005 learned_addr learned my addr 172.21.15.112:0/2215956005 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35147640 1 -- 172.21.15.112:0/2215956005 >> v2:172.21.15.187:3300/0 conn(0x7efc30074590 msgr2=0x7efc300b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35948640 1 --2- 172.21.15.112:0/2215956005 >> v2:172.21.15.187:3300/0 conn(0x7efc30074590 0x7efc300b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35147640 1 --2- 172.21.15.112:0/2215956005 >> v2:172.21.15.187:3300/0 conn(0x7efc30074590 0x7efc300b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc34946640 1 -- 172.21.15.112:0/2215956005 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1361593245 0 0) 0x7efc3009bec0 con 0x7efc301427e0 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35147640 1 -- 172.21.15.112:0/2215956005 >> v1:172.21.15.112:6789/0 conn(0x7efc301427e0 legacy=0x7efc3006c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35147640 1 -- 172.21.15.112:0/2215956005 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3009c9c0 con 0x7efc300b3a80 2024-04-25T00:55:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.044+0000 7efc35147640 1 --2- 172.21.15.112:0/2215956005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301422a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc300b0290 tx=0x7efc2c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194d17dd437f9a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc34946640 1 -- 172.21.15.112:0/2215956005 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0b2070 con 0x7efc300b3a80 2024-04-25T00:55:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc34946640 1 -- 172.21.15.112:0/2215956005 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2c0ad680 con 0x7efc300b3a80 2024-04-25T00:55:46.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc34946640 1 -- 172.21.15.112:0/2215956005 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0ada50 con 0x7efc300b3a80 2024-04-25T00:55:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- 172.21.15.112:0/2215956005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 msgr2=0x7efc301422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 --2- 172.21.15.112:0/2215956005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301422a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efc300b0290 tx=0x7efc2c01c460 comp rx=0 tx=0).stop 2024-04-25T00:55:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- 172.21.15.112:0/2215956005 shutdown_connections 2024-04-25T00:55:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 --2- 172.21.15.112:0/2215956005 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301422a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 --2- 172.21.15.112:0/2215956005 >> v2:172.21.15.187:3300/0 conn(0x7efc30074590 0x7efc300b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- 172.21.15.112:0/2215956005 >> 172.21.15.112:0/2215956005 conn(0x7efc300b9700 msgr2=0x7efc300b9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- 172.21.15.112:0/2215956005 shutdown_connections 2024-04-25T00:55:46.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- 172.21.15.112:0/2215956005 wait complete. 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 Processor -- start 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- start start 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc30074590 0x7efc3013f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301e3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc301427e0 0x7efc301e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7efc300b40d0 con 0x7efc300b3a80 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7efc300b3f50 con 0x7efc30074590 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7efc30142c10 con 0x7efc301427e0 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301e3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51358/0 (socket says 172.21.15.112:51358) 2024-04-25T00:55:46.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 -- 172.21.15.112:0/3423973124 learned_addr learned my addr 172.21.15.112:0/3423973124 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35948640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc30074590 0x7efc3013f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc36149640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc301427e0 0x7efc301e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 -- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc301427e0 msgr2=0x7efc301e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc301427e0 0x7efc301e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 -- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc30074590 msgr2=0x7efc3013f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc30074590 0x7efc3013f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 -- 172.21.15.112:0/3423973124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3013e650 con 0x7efc300b3a80 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc36149640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc301427e0 0x7efc301e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc35147640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301e3610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efc2c01a090 tx=0x7efc2c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc267fc640 1 -- 172.21.15.112:0/3423973124 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0b2040 con 0x7efc300b3a80 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc267fc640 1 -- 172.21.15.112:0/3423973124 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2c0adbf0 con 0x7efc300b3a80 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3013c510 con 0x7efc300b3a80 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc267fc640 1 -- 172.21.15.112:0/3423973124 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c004030 con 0x7efc300b3a80 2024-04-25T00:55:46.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc300afb50 con 0x7efc300b3a80 2024-04-25T00:55:46.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.048+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3004f580 con 0x7efc300b3a80 2024-04-25T00:55:46.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.052+0000 7efc267fc640 1 -- 172.21.15.112:0/3423973124 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efc2c0041d0 con 0x7efc300b3a80 2024-04-25T00:55:46.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.052+0000 7efc267fc640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7efc140678c0 0x7efc14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.052+0000 7efc267fc640 1 -- 172.21.15.112:0/3423973124 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efc2c11f990 con 0x7efc300b3a80 2024-04-25T00:55:46.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.052+0000 7efc35948640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7efc140678c0 0x7efc14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.052+0000 7efc35948640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7efc140678c0 0x7efc14069d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efc3013ceb0 tx=0x7efc20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.056+0000 7efc267fc640 1 -- 172.21.15.112:0/3423973124 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efc2c0ba020 con 0x7efc300b3a80 2024-04-25T00:55:46.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.200+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc3013e8c0 con 0x7efc140678c0 2024-04-25T00:55:46.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.208+0000 7efc267fc640 1 -- 172.21.15.112:0/3423973124 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7efc3013e8c0 con 0x7efc140678c0 2024-04-25T00:55:46.213 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:46.215 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7efc140678c0 msgr2=0x7efc14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7efc140678c0 0x7efc14069d80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efc3013ceb0 tx=0x7efc20008040 comp rx=0 tx=0).stop 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 msgr2=0x7efc301e3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301e3610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efc2c01a090 tx=0x7efc2c019200 comp rx=0 tx=0).stop 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 shutdown_connections 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7efc301427e0 0x7efc301e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efc300b3a80 0x7efc301e3610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7efc140678c0 0x7efc14069d80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 --2- 172.21.15.112:0/3423973124 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7efc30074590 0x7efc3013f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 >> 172.21.15.112:0/3423973124 conn(0x7efc300b9700 msgr2=0x7efc300b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 shutdown_connections 2024-04-25T00:55:46.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.212+0000 7efc37bac640 1 -- 172.21.15.112:0/3423973124 wait complete. 2024-04-25T00:55:46.231 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T00:55:45.516289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.932808"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018125+0000","last_change":"2024-04-25T00:55:39.661241+0000","last_active":"2024-04-25T00:55:41.018125+0000","last_peered":"2024-04-25T00:55:41.018125+0000","last_clean":"2024-04-25T00:55:41.018125+0000","last_became_active":"2024-04-25T00:55:39.660150+0000","last_became_peered":"2024-04-25T00:55:39.660150+0000","last_unstale":"2024-04-25T00:55:41.018125+0000","last_undegraded":"2024-04-25T00:55:41.018125+0000","last_fullsized":"2024-04-25T00:55:41.018125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:11.034367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637050+0000","last_change":"2024-04-25T00:55:39.633699+0000","last_active":"2024-04-25T00:55:40.637050+0000","last_peered":"2024-04-25T00:55:40.637050+0000","last_clean":"2024-04-25T00:55:40.637050+0000","last_became_active":"2024-04-25T00:55:39.632942+0000","last_became_peered":"2024-04-25T00:55:39.632942+0000","last_unstale":"2024-04-25T00:55:40.637050+0000","last_undegraded":"2024-04-25T00:55:40.637050+0000","last_fullsized":"2024-04-25T00:55:40.637050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018670+0000","last_change":"2024-04-25T00:55:39.629762+0000","last_active":"2024-04-25T00:55:41.018670+0000","last_peered":"2024-04-25T00:55:41.018670+0000","last_clean":"2024-04-25T00:55:41.018670+0000","last_became_active":"2024-04-25T00:55:39.628798+0000","last_became_peered":"2024-04-25T00:55:39.628798+0000","last_unstale":"2024-04-25T00:55:41.018670+0000","last_undegraded":"2024-04-25T00:55:41.018670+0000","last_fullsized":"2024-04-25T00:55:41.018670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:23.939072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637845+0000","last_change":"2024-04-25T00:55:39.643398+0000","last_active":"2024-04-25T00:55:40.637845+0000","last_peered":"2024-04-25T00:55:40.637845+0000","last_clean":"2024-04-25T00:55:40.637845+0000","last_became_active":"2024-04-25T00:55:39.642844+0000","last_became_peered":"2024-04-25T00:55:39.642844+0000","last_unstale":"2024-04-25T00:55:40.637845+0000","last_undegraded":"2024-04-25T00:55:40.637845+0000","last_fullsized":"2024-04-25T00:55:40.637845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017483+0000","last_change":"2024-04-25T00:55:39.649899+0000","last_active":"2024-04-25T00:55:41.017483+0000","last_peered":"2024-04-25T00:55:41.017483+0000","last_clean":"2024-04-25T00:55:41.017483+0000","last_became_active":"2024-04-25T00:55:39.648852+0000","last_became_peered":"2024-04-25T00:55:39.648852+0000","last_unstale":"2024-04-25T00:55:41.017483+0000","last_undegraded":"2024-04-25T00:55:41.017483+0000","last_fullsized":"2024-04-25T00:55:41.017483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638862+0000","last_change":"2024-04-25T00:55:39.635369+0000","last_active":"2024-04-25T00:55:40.638862+0000","last_peered":"2024-04-25T00:55:40.638862+0000","last_clean":"2024-04-25T00:55:40.638862+0000","last_became_active":"2024-04-25T00:55:39.634678+0000","last_became_peered":"2024-04-25T00:55:39.634678+0000","last_unstale":"2024-04-25T00:55:40.638862+0000","last_undegraded":"2024-04-25T00:55:40.638862+0000","last_fullsized":"2024-04-25T00:55:40.638862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:30.556421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016962+0000","last_change":"2024-04-25T00:55:39.646309+0000","last_active":"2024-04-25T00:55:41.016962+0000","last_peered":"2024-04-25T00:55:41.016962+0000","last_clean":"2024-04-25T00:55:41.016962+0000","last_became_active":"2024-04-25T00:55:39.645778+0000","last_became_peered":"2024-04-25T00:55:39.645778+0000","last_unstale":"2024-04-25T00:55:41.016962+0000","last_undegraded":"2024-04-25T00:55:41.016962+0000","last_fullsized":"2024-04-25T00:55:41.016962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020590+0000","last_change":"2024-04-25T00:55:39.643250+0000","last_active":"2024-04-25T00:55:41.020590+0000","last_peered":"2024-04-25T00:55:41.020590+0000","last_clean":"2024-04-25T00:55:41.020590+0000","last_became_active":"2024-04-25T00:55:39.642948+0000","last_became_peered":"2024-04-25T00:55:39.642948+0000","last_unstale":"2024-04-25T00:55:41.020590+0000","last_undegraded":"2024-04-25T00:55:41.020590+0000","last_fullsized":"2024-04-25T00:55:41.020590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:53.762656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018212+0000","last_change":"2024-04-25T00:55:39.658665+0000","last_active":"2024-04-25T00:55:41.018212+0000","last_peered":"2024-04-25T00:55:41.018212+0000","last_clean":"2024-04-25T00:55:41.018212+0000","last_became_active":"2024-04-25T00:55:39.658276+0000","last_became_peered":"2024-04-25T00:55:39.658276+0000","last_unstale":"2024-04-25T00:55:41.018212+0000","last_undegraded":"2024-04-25T00:55:41.018212+0000","last_fullsized":"2024-04-25T00:55:41.018212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:47.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652697+0000","last_change":"2024-04-25T00:55:39.660203+0000","last_active":"2024-04-25T00:55:40.652697+0000","last_peered":"2024-04-25T00:55:40.652697+0000","last_clean":"2024-04-25T00:55:40.652697+0000","last_became_active":"2024-04-25T00:55:39.656617+0000","last_became_peered":"2024-04-25T00:55:39.656617+0000","last_unstale":"2024-04-25T00:55:40.652697+0000","last_undegraded":"2024-04-25T00:55:40.652697+0000","last_fullsized":"2024-04-25T00:55:40.652697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652875+0000","last_change":"2024-04-25T00:55:39.638495+0000","last_active":"2024-04-25T00:55:40.652875+0000","last_peered":"2024-04-25T00:55:40.652875+0000","last_clean":"2024-04-25T00:55:40.652875+0000","last_became_active":"2024-04-25T00:55:39.631516+0000","last_became_peered":"2024-04-25T00:55:39.631516+0000","last_unstale":"2024-04-25T00:55:40.652875+0000","last_undegraded":"2024-04-25T00:55:40.652875+0000","last_fullsized":"2024-04-25T00:55:40.652875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636317+0000","last_change":"2024-04-25T00:55:39.657591+0000","last_active":"2024-04-25T00:55:40.636317+0000","last_peered":"2024-04-25T00:55:40.636317+0000","last_clean":"2024-04-25T00:55:40.636317+0000","last_became_active":"2024-04-25T00:55:39.657259+0000","last_became_peered":"2024-04-25T00:55:39.657259+0000","last_unstale":"2024-04-25T00:55:40.636317+0000","last_undegraded":"2024-04-25T00:55:40.636317+0000","last_fullsized":"2024-04-25T00:55:40.636317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:56.786582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.022200+0000","last_change":"2024-04-25T00:55:39.656495+0000","last_active":"2024-04-25T00:55:41.022200+0000","last_peered":"2024-04-25T00:55:41.022200+0000","last_clean":"2024-04-25T00:55:41.022200+0000","last_became_active":"2024-04-25T00:55:39.655994+0000","last_became_peered":"2024-04-25T00:55:39.655994+0000","last_unstale":"2024-04-25T00:55:41.022200+0000","last_undegraded":"2024-04-25T00:55:41.022200+0000","last_fullsized":"2024-04-25T00:55:41.022200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019282+0000","last_change":"2024-04-25T00:55:39.636352+0000","last_active":"2024-04-25T00:55:41.019282+0000","last_peered":"2024-04-25T00:55:41.019282+0000","last_clean":"2024-04-25T00:55:41.019282+0000","last_became_active":"2024-04-25T00:55:39.636013+0000","last_became_peered":"2024-04-25T00:55:39.636013+0000","last_unstale":"2024-04-25T00:55:41.019282+0000","last_undegraded":"2024-04-25T00:55:41.019282+0000","last_fullsized":"2024-04-25T00:55:41.019282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636741+0000","last_change":"2024-04-25T00:55:39.634943+0000","last_active":"2024-04-25T00:55:40.636741+0000","last_peered":"2024-04-25T00:55:40.636741+0000","last_clean":"2024-04-25T00:55:40.636741+0000","last_became_active":"2024-04-25T00:55:39.634116+0000","last_became_peered":"2024-04-25T00:55:39.634116+0000","last_unstale":"2024-04-25T00:55:40.636741+0000","last_undegraded":"2024-04-25T00:55:40.636741+0000","last_fullsized":"2024-04-25T00:55:40.636741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637210+0000","last_change":"2024-04-25T00:55:39.630149+0000","last_active":"2024-04-25T00:55:40.637210+0000","last_peered":"2024-04-25T00:55:40.637210+0000","last_clean":"2024-04-25T00:55:40.637210+0000","last_became_active":"2024-04-25T00:55:39.629387+0000","last_became_peered":"2024-04-25T00:55:39.629387+0000","last_unstale":"2024-04-25T00:55:40.637210+0000","last_undegraded":"2024-04-25T00:55:40.637210+0000","last_fullsized":"2024-04-25T00:55:40.637210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638541+0000","last_change":"2024-04-25T00:55:39.650995+0000","last_active":"2024-04-25T00:55:40.638541+0000","last_peered":"2024-04-25T00:55:40.638541+0000","last_clean":"2024-04-25T00:55:40.638541+0000","last_became_active":"2024-04-25T00:55:39.650711+0000","last_became_peered":"2024-04-25T00:55:39.650711+0000","last_unstale":"2024-04-25T00:55:40.638541+0000","last_undegraded":"2024-04-25T00:55:40.638541+0000","last_fullsized":"2024-04-25T00:55:40.638541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.187193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019011+0000","last_change":"2024-04-25T00:55:39.656754+0000","last_active":"2024-04-25T00:55:41.019011+0000","last_peered":"2024-04-25T00:55:41.019011+0000","last_clean":"2024-04-25T00:55:41.019011+0000","last_became_active":"2024-04-25T00:55:39.656397+0000","last_became_peered":"2024-04-25T00:55:39.656397+0000","last_unstale":"2024-04-25T00:55:41.019011+0000","last_undegraded":"2024-04-25T00:55:41.019011+0000","last_fullsized":"2024-04-25T00:55:41.019011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.087648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018205+0000","last_change":"2024-04-25T00:55:39.648560+0000","last_active":"2024-04-25T00:55:41.018205+0000","last_peered":"2024-04-25T00:55:41.018205+0000","last_clean":"2024-04-25T00:55:41.018205+0000","last_became_active":"2024-04-25T00:55:39.647080+0000","last_became_peered":"2024-04-25T00:55:39.647080+0000","last_unstale":"2024-04-25T00:55:41.018205+0000","last_undegraded":"2024-04-25T00:55:41.018205+0000","last_fullsized":"2024-04-25T00:55:41.018205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:22.249474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.653020+0000","last_change":"2024-04-25T00:55:39.659058+0000","last_active":"2024-04-25T00:55:40.653020+0000","last_peered":"2024-04-25T00:55:40.653020+0000","last_clean":"2024-04-25T00:55:40.653020+0000","last_became_active":"2024-04-25T00:55:39.657152+0000","last_became_peered":"2024-04-25T00:55:39.657152+0000","last_unstale":"2024-04-25T00:55:40.653020+0000","last_undegraded":"2024-04-25T00:55:40.653020+0000","last_fullsized":"2024-04-25T00:55:40.653020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020095+0000","last_change":"2024-04-25T00:55:39.650890+0000","last_active":"2024-04-25T00:55:41.020095+0000","last_peered":"2024-04-25T00:55:41.020095+0000","last_clean":"2024-04-25T00:55:41.020095+0000","last_became_active":"2024-04-25T00:55:39.650235+0000","last_became_peered":"2024-04-25T00:55:39.650235+0000","last_unstale":"2024-04-25T00:55:41.020095+0000","last_undegraded":"2024-04-25T00:55:41.020095+0000","last_fullsized":"2024-04-25T00:55:41.020095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638004+0000","last_change":"2024-04-25T00:55:39.646725+0000","last_active":"2024-04-25T00:55:40.638004+0000","last_peered":"2024-04-25T00:55:40.638004+0000","last_clean":"2024-04-25T00:55:40.638004+0000","last_became_active":"2024-04-25T00:55:39.646192+0000","last_became_peered":"2024-04-25T00:55:39.646192+0000","last_unstale":"2024-04-25T00:55:40.638004+0000","last_undegraded":"2024-04-25T00:55:40.638004+0000","last_fullsized":"2024-04-25T00:55:40.638004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:51.008333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020594+0000","last_change":"2024-04-25T00:55:39.642303+0000","last_active":"2024-04-25T00:55:41.020594+0000","last_peered":"2024-04-25T00:55:41.020594+0000","last_clean":"2024-04-25T00:55:41.020594+0000","last_became_active":"2024-04-25T00:55:39.641627+0000","last_became_peered":"2024-04-25T00:55:39.641627+0000","last_unstale":"2024-04-25T00:55:41.020594+0000","last_undegraded":"2024-04-25T00:55:41.020594+0000","last_fullsized":"2024-04-25T00:55:41.020594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637271+0000","last_change":"2024-04-25T00:55:39.643919+0000","last_active":"2024-04-25T00:55:40.637271+0000","last_peered":"2024-04-25T00:55:40.637271+0000","last_clean":"2024-04-25T00:55:40.637271+0000","last_became_active":"2024-04-25T00:55:39.643196+0000","last_became_peered":"2024-04-25T00:55:39.643196+0000","last_unstale":"2024-04-25T00:55:40.637271+0000","last_undegraded":"2024-04-25T00:55:40.637271+0000","last_fullsized":"2024-04-25T00:55:40.637271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021581+0000","last_change":"2024-04-25T00:55:39.641462+0000","last_active":"2024-04-25T00:55:41.021581+0000","last_peered":"2024-04-25T00:55:41.021581+0000","last_clean":"2024-04-25T00:55:41.021581+0000","last_became_active":"2024-04-25T00:55:39.640648+0000","last_became_peered":"2024-04-25T00:55:39.640648+0000","last_unstale":"2024-04-25T00:55:41.021581+0000","last_undegraded":"2024-04-25T00:55:41.021581+0000","last_fullsized":"2024-04-25T00:55:41.021581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652767+0000","last_change":"2024-04-25T00:55:39.648816+0000","last_active":"2024-04-25T00:55:40.652767+0000","last_peered":"2024-04-25T00:55:40.652767+0000","last_clean":"2024-04-25T00:55:40.652767+0000","last_became_active":"2024-04-25T00:55:39.648337+0000","last_became_peered":"2024-04-25T00:55:39.648337+0000","last_unstale":"2024-04-25T00:55:40.652767+0000","last_undegraded":"2024-04-25T00:55:40.652767+0000","last_fullsized":"2024-04-25T00:55:40.652767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637430+0000","last_change":"2024-04-25T00:55:39.644333+0000","last_active":"2024-04-25T00:55:40.637430+0000","last_peered":"2024-04-25T00:55:40.637430+0000","last_clean":"2024-04-25T00:55:40.637430+0000","last_became_active":"2024-04-25T00:55:39.643706+0000","last_became_peered":"2024-04-25T00:55:39.643706+0000","last_unstale":"2024-04-25T00:55:40.637430+0000","last_undegraded":"2024-04-25T00:55:40.637430+0000","last_fullsized":"2024-04-25T00:55:40.637430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021853+0000","last_change":"2024-04-25T00:55:39.631560+0000","last_active":"2024-04-25T00:55:41.021853+0000","last_peered":"2024-04-25T00:55:41.021853+0000","last_clean":"2024-04-25T00:55:41.021853+0000","last_became_active":"2024-04-25T00:55:39.630796+0000","last_became_peered":"2024-04-25T00:55:39.630796+0000","last_unstale":"2024-04-25T00:55:41.021853+0000","last_undegraded":"2024-04-25T00:55:41.021853+0000","last_fullsized":"2024-04-25T00:55:41.021853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020172+0000","last_change":"2024-04-25T00:55:39.643191+0000","last_active":"2024-04-25T00:55:41.020172+0000","last_peered":"2024-04-25T00:55:41.020172+0000","last_clean":"2024-04-25T00:55:41.020172+0000","last_became_active":"2024-04-25T00:55:39.642846+0000","last_became_peered":"2024-04-25T00:55:39.642846+0000","last_unstale":"2024-04-25T00:55:41.020172+0000","last_undegraded":"2024-04-25T00:55:41.020172+0000","last_fullsized":"2024-04-25T00:55:41.020172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638416+0000","last_change":"2024-04-25T00:55:39.643174+0000","last_active":"2024-04-25T00:55:40.638416+0000","last_peered":"2024-04-25T00:55:40.638416+0000","last_clean":"2024-04-25T00:55:40.638416+0000","last_became_active":"2024-04-25T00:55:39.642575+0000","last_became_peered":"2024-04-25T00:55:39.642575+0000","last_unstale":"2024-04-25T00:55:40.638416+0000","last_undegraded":"2024-04-25T00:55:40.638416+0000","last_fullsized":"2024-04-25T00:55:40.638416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:47.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018168+0000","last_change":"2024-04-25T00:55:39.627384+0000","last_active":"2024-04-25T00:55:41.018168+0000","last_peered":"2024-04-25T00:55:41.018168+0000","last_clean":"2024-04-25T00:55:41.018168+0000","last_became_active":"2024-04-25T00:55:39.627078+0000","last_became_peered":"2024-04-25T00:55:39.627078+0000","last_unstale":"2024-04-25T00:55:41.018168+0000","last_undegraded":"2024-04-25T00:55:41.018168+0000","last_fullsized":"2024-04-25T00:55:41.018168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:21.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652573+0000","last_change":"2024-04-25T00:55:39.655121+0000","last_active":"2024-04-25T00:55:40.652573+0000","last_peered":"2024-04-25T00:55:40.652573+0000","last_clean":"2024-04-25T00:55:40.652573+0000","last_became_active":"2024-04-25T00:55:39.654774+0000","last_became_peered":"2024-04-25T00:55:39.654774+0000","last_unstale":"2024-04-25T00:55:40.652573+0000","last_undegraded":"2024-04-25T00:55:40.652573+0000","last_fullsized":"2024-04-25T00:55:40.652573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636316+0000","last_change":"2024-04-25T00:55:39.634737+0000","last_active":"2024-04-25T00:55:40.636316+0000","last_peered":"2024-04-25T00:55:40.636316+0000","last_clean":"2024-04-25T00:55:40.636316+0000","last_became_active":"2024-04-25T00:55:39.633943+0000","last_became_peered":"2024-04-25T00:55:39.633943+0000","last_unstale":"2024-04-25T00:55:40.636316+0000","last_undegraded":"2024-04-25T00:55:40.636316+0000","last_fullsized":"2024-04-25T00:55:40.636316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:09.262927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021945+0000","last_change":"2024-04-25T00:55:38.629602+0000","last_active":"2024-04-25T00:55:41.021945+0000","last_peered":"2024-04-25T00:55:41.021945+0000","last_clean":"2024-04-25T00:55:41.021945+0000","last_became_active":"2024-04-25T00:55:38.628956+0000","last_became_peered":"2024-04-25T00:55:38.628956+0000","last_unstale":"2024-04-25T00:55:41.021945+0000","last_undegraded":"2024-04-25T00:55:41.021945+0000","last_fullsized":"2024-04-25T00:55:41.021945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:50.369672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020489+0000","last_change":"2024-04-25T00:55:39.630640+0000","last_active":"2024-04-25T00:55:41.020489+0000","last_peered":"2024-04-25T00:55:41.020489+0000","last_clean":"2024-04-25T00:55:41.020489+0000","last_became_active":"2024-04-25T00:55:39.629906+0000","last_became_peered":"2024-04-25T00:55:39.629906+0000","last_unstale":"2024-04-25T00:55:41.020489+0000","last_undegraded":"2024-04-25T00:55:41.020489+0000","last_fullsized":"2024-04-25T00:55:41.020489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652654+0000","last_change":"2024-04-25T00:55:38.626215+0000","last_active":"2024-04-25T00:55:40.652654+0000","last_peered":"2024-04-25T00:55:40.652654+0000","last_clean":"2024-04-25T00:55:40.652654+0000","last_became_active":"2024-04-25T00:55:38.625439+0000","last_became_peered":"2024-04-25T00:55:38.625439+0000","last_unstale":"2024-04-25T00:55:40.652654+0000","last_undegraded":"2024-04-25T00:55:40.652654+0000","last_fullsized":"2024-04-25T00:55:40.652654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020655+0000","last_change":"2024-04-25T00:55:38.626401+0000","last_active":"2024-04-25T00:55:41.020655+0000","last_peered":"2024-04-25T00:55:41.020655+0000","last_clean":"2024-04-25T00:55:41.020655+0000","last_became_active":"2024-04-25T00:55:38.625884+0000","last_became_peered":"2024-04-25T00:55:38.625884+0000","last_unstale":"2024-04-25T00:55:41.020655+0000","last_undegraded":"2024-04-25T00:55:41.020655+0000","last_fullsized":"2024-04-25T00:55:41.020655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020981+0000","last_change":"2024-04-25T00:55:39.638181+0000","last_active":"2024-04-25T00:55:41.020981+0000","last_peered":"2024-04-25T00:55:41.020981+0000","last_clean":"2024-04-25T00:55:41.020981+0000","last_became_active":"2024-04-25T00:55:39.637900+0000","last_became_peered":"2024-04-25T00:55:39.637900+0000","last_unstale":"2024-04-25T00:55:41.020981+0000","last_undegraded":"2024-04-25T00:55:41.020981+0000","last_fullsized":"2024-04-25T00:55:41.020981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637316+0000","last_change":"2024-04-25T00:55:38.621335+0000","last_active":"2024-04-25T00:55:40.637316+0000","last_peered":"2024-04-25T00:55:40.637316+0000","last_clean":"2024-04-25T00:55:40.637316+0000","last_became_active":"2024-04-25T00:55:38.620467+0000","last_became_peered":"2024-04-25T00:55:38.620467+0000","last_unstale":"2024-04-25T00:55:40.637316+0000","last_undegraded":"2024-04-25T00:55:40.637316+0000","last_fullsized":"2024-04-25T00:55:40.637316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636806+0000","last_change":"2024-04-25T00:55:39.642780+0000","last_active":"2024-04-25T00:55:40.636806+0000","last_peered":"2024-04-25T00:55:40.636806+0000","last_clean":"2024-04-25T00:55:40.636806+0000","last_became_active":"2024-04-25T00:55:39.642410+0000","last_became_peered":"2024-04-25T00:55:39.642410+0000","last_unstale":"2024-04-25T00:55:40.636806+0000","last_undegraded":"2024-04-25T00:55:40.636806+0000","last_fullsized":"2024-04-25T00:55:40.636806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652622+0000","last_change":"2024-04-25T00:55:38.634413+0000","last_active":"2024-04-25T00:55:40.652622+0000","last_peered":"2024-04-25T00:55:40.652622+0000","last_clean":"2024-04-25T00:55:40.652622+0000","last_became_active":"2024-04-25T00:55:38.633760+0000","last_became_peered":"2024-04-25T00:55:38.633760+0000","last_unstale":"2024-04-25T00:55:40.652622+0000","last_undegraded":"2024-04-25T00:55:40.652622+0000","last_fullsized":"2024-04-25T00:55:40.652622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637854+0000","last_change":"2024-04-25T00:55:39.644911+0000","last_active":"2024-04-25T00:55:40.637854+0000","last_peered":"2024-04-25T00:55:40.637854+0000","last_clean":"2024-04-25T00:55:40.637854+0000","last_became_active":"2024-04-25T00:55:39.644568+0000","last_became_peered":"2024-04-25T00:55:39.644568+0000","last_unstale":"2024-04-25T00:55:40.637854+0000","last_undegraded":"2024-04-25T00:55:40.637854+0000","last_fullsized":"2024-04-25T00:55:40.637854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:16.517320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637925+0000","last_change":"2024-04-25T00:55:38.637548+0000","last_active":"2024-04-25T00:55:40.637925+0000","last_peered":"2024-04-25T00:55:40.637925+0000","last_clean":"2024-04-25T00:55:40.637925+0000","last_became_active":"2024-04-25T00:55:38.636962+0000","last_became_peered":"2024-04-25T00:55:38.636962+0000","last_unstale":"2024-04-25T00:55:40.637925+0000","last_undegraded":"2024-04-25T00:55:40.637925+0000","last_fullsized":"2024-04-25T00:55:40.637925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020866+0000","last_change":"2024-04-25T00:55:39.628306+0000","last_active":"2024-04-25T00:55:41.020866+0000","last_peered":"2024-04-25T00:55:41.020866+0000","last_clean":"2024-04-25T00:55:41.020866+0000","last_became_active":"2024-04-25T00:55:39.627530+0000","last_became_peered":"2024-04-25T00:55:39.627530+0000","last_unstale":"2024-04-25T00:55:41.020866+0000","last_undegraded":"2024-04-25T00:55:41.020866+0000","last_fullsized":"2024-04-25T00:55:41.020866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637335+0000","last_change":"2024-04-25T00:55:38.640365+0000","last_active":"2024-04-25T00:55:40.637335+0000","last_peered":"2024-04-25T00:55:40.637335+0000","last_clean":"2024-04-25T00:55:40.637335+0000","last_became_active":"2024-04-25T00:55:38.639988+0000","last_became_peered":"2024-04-25T00:55:38.639988+0000","last_unstale":"2024-04-25T00:55:40.637335+0000","last_undegraded":"2024-04-25T00:55:40.637335+0000","last_fullsized":"2024-04-25T00:55:40.637335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636444+0000","last_change":"2024-04-25T00:55:39.647034+0000","last_active":"2024-04-25T00:55:40.636444+0000","last_peered":"2024-04-25T00:55:40.636444+0000","last_clean":"2024-04-25T00:55:40.636444+0000","last_became_active":"2024-04-25T00:55:39.644279+0000","last_became_peered":"2024-04-25T00:55:39.644279+0000","last_unstale":"2024-04-25T00:55:40.636444+0000","last_undegraded":"2024-04-25T00:55:40.636444+0000","last_fullsized":"2024-04-25T00:55:40.636444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:21.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652840+0000","last_change":"2024-04-25T00:55:38.626291+0000","last_active":"2024-04-25T00:55:40.652840+0000","last_peered":"2024-04-25T00:55:40.652840+0000","last_clean":"2024-04-25T00:55:40.652840+0000","last_became_active":"2024-04-25T00:55:38.625530+0000","last_became_peered":"2024-04-25T00:55:38.625530+0000","last_unstale":"2024-04-25T00:55:40.652840+0000","last_undegraded":"2024-04-25T00:55:40.652840+0000","last_fullsized":"2024-04-25T00:55:40.652840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017484+0000","last_change":"2024-04-25T00:55:39.649518+0000","last_active":"2024-04-25T00:55:41.017484+0000","last_peered":"2024-04-25T00:55:41.017484+0000","last_clean":"2024-04-25T00:55:41.017484+0000","last_became_active":"2024-04-25T00:55:39.648701+0000","last_became_peered":"2024-04-25T00:55:39.648701+0000","last_unstale":"2024-04-25T00:55:41.017484+0000","last_undegraded":"2024-04-25T00:55:41.017484+0000","last_fullsized":"2024-04-25T00:55:41.017484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.552805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020445+0000","last_change":"2024-04-25T00:55:38.610242+0000","last_active":"2024-04-25T00:55:41.020445+0000","last_peered":"2024-04-25T00:55:41.020445+0000","last_clean":"2024-04-25T00:55:41.020445+0000","last_became_active":"2024-04-25T00:55:38.609855+0000","last_became_peered":"2024-04-25T00:55:38.609855+0000","last_unstale":"2024-04-25T00:55:41.020445+0000","last_undegraded":"2024-04-25T00:55:41.020445+0000","last_fullsized":"2024-04-25T00:55:41.020445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019689+0000","last_change":"2024-04-25T00:55:39.646421+0000","last_active":"2024-04-25T00:55:41.019689+0000","last_peered":"2024-04-25T00:55:41.019689+0000","last_clean":"2024-04-25T00:55:41.019689+0000","last_became_active":"2024-04-25T00:55:39.646119+0000","last_became_peered":"2024-04-25T00:55:39.646119+0000","last_unstale":"2024-04-25T00:55:41.019689+0000","last_undegraded":"2024-04-25T00:55:41.019689+0000","last_fullsized":"2024-04-25T00:55:41.019689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638593+0000","last_change":"2024-04-25T00:55:38.632875+0000","last_active":"2024-04-25T00:55:40.638593+0000","last_peered":"2024-04-25T00:55:40.638593+0000","last_clean":"2024-04-25T00:55:40.638593+0000","last_became_active":"2024-04-25T00:55:38.631370+0000","last_became_peered":"2024-04-25T00:55:38.631370+0000","last_unstale":"2024-04-25T00:55:40.638593+0000","last_undegraded":"2024-04-25T00:55:40.638593+0000","last_fullsized":"2024-04-25T00:55:40.638593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017573+0000","last_change":"2024-04-25T00:55:39.661000+0000","last_active":"2024-04-25T00:55:41.017573+0000","last_peered":"2024-04-25T00:55:41.017573+0000","last_clean":"2024-04-25T00:55:41.017573+0000","last_became_active":"2024-04-25T00:55:39.660704+0000","last_became_peered":"2024-04-25T00:55:39.660704+0000","last_unstale":"2024-04-25T00:55:41.017573+0000","last_undegraded":"2024-04-25T00:55:41.017573+0000","last_fullsized":"2024-04-25T00:55:41.017573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:16.517320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652441+0000","last_change":"2024-04-25T00:55:38.614039+0000","last_active":"2024-04-25T00:55:40.652441+0000","last_peered":"2024-04-25T00:55:40.652441+0000","last_clean":"2024-04-25T00:55:40.652441+0000","last_became_active":"2024-04-25T00:55:38.612513+0000","last_became_peered":"2024-04-25T00:55:38.612513+0000","last_unstale":"2024-04-25T00:55:40.652441+0000","last_undegraded":"2024-04-25T00:55:40.652441+0000","last_fullsized":"2024-04-25T00:55:40.652441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652439+0000","last_change":"2024-04-25T00:55:39.639974+0000","last_active":"2024-04-25T00:55:40.652439+0000","last_peered":"2024-04-25T00:55:40.652439+0000","last_clean":"2024-04-25T00:55:40.652439+0000","last_became_active":"2024-04-25T00:55:39.639397+0000","last_became_peered":"2024-04-25T00:55:39.639397+0000","last_unstale":"2024-04-25T00:55:40.652439+0000","last_undegraded":"2024-04-25T00:55:40.652439+0000","last_fullsized":"2024-04-25T00:55:40.652439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017966+0000","last_change":"2024-04-25T00:55:38.601709+0000","last_active":"2024-04-25T00:55:41.017966+0000","last_peered":"2024-04-25T00:55:41.017966+0000","last_clean":"2024-04-25T00:55:41.017966+0000","last_became_active":"2024-04-25T00:55:38.601079+0000","last_became_peered":"2024-04-25T00:55:38.601079+0000","last_unstale":"2024-04-25T00:55:41.017966+0000","last_undegraded":"2024-04-25T00:55:41.017966+0000","last_fullsized":"2024-04-25T00:55:41.017966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:50.369672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016947+0000","last_change":"2024-04-25T00:55:39.652284+0000","last_active":"2024-04-25T00:55:41.016947+0000","last_peered":"2024-04-25T00:55:41.016947+0000","last_clean":"2024-04-25T00:55:41.016947+0000","last_became_active":"2024-04-25T00:55:39.651442+0000","last_became_peered":"2024-04-25T00:55:39.651442+0000","last_unstale":"2024-04-25T00:55:41.016947+0000","last_undegraded":"2024-04-25T00:55:41.016947+0000","last_fullsized":"2024-04-25T00:55:41.016947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636948+0000","last_change":"2024-04-25T00:55:38.603875+0000","last_active":"2024-04-25T00:55:40.636948+0000","last_peered":"2024-04-25T00:55:40.636948+0000","last_clean":"2024-04-25T00:55:40.636948+0000","last_became_active":"2024-04-25T00:55:38.602412+0000","last_became_peered":"2024-04-25T00:55:38.602412+0000","last_unstale":"2024-04-25T00:55:40.636948+0000","last_undegraded":"2024-04-25T00:55:40.636948+0000","last_fullsized":"2024-04-25T00:55:40.636948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:34.488773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018026+0000","last_change":"2024-04-25T00:55:39.651592+0000","last_active":"2024-04-25T00:55:41.018026+0000","last_peered":"2024-04-25T00:55:41.018026+0000","last_clean":"2024-04-25T00:55:41.018026+0000","last_became_active":"2024-04-25T00:55:39.651045+0000","last_became_peered":"2024-04-25T00:55:39.651045+0000","last_unstale":"2024-04-25T00:55:41.018026+0000","last_undegraded":"2024-04-25T00:55:41.018026+0000","last_fullsized":"2024-04-25T00:55:41.018026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:51.008333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018836+0000","last_change":"2024-04-25T00:55:38.613063+0000","last_active":"2024-04-25T00:55:41.018836+0000","last_peered":"2024-04-25T00:55:41.018836+0000","last_clean":"2024-04-25T00:55:41.018836+0000","last_became_active":"2024-04-25T00:55:38.612651+0000","last_became_peered":"2024-04-25T00:55:38.612651+0000","last_unstale":"2024-04-25T00:55:41.018836+0000","last_undegraded":"2024-04-25T00:55:41.018836+0000","last_fullsized":"2024-04-25T00:55:41.018836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:25.842602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018376+0000","last_change":"2024-04-25T00:55:39.661043+0000","last_active":"2024-04-25T00:55:41.018376+0000","last_peered":"2024-04-25T00:55:41.018376+0000","last_clean":"2024-04-25T00:55:41.018376+0000","last_became_active":"2024-04-25T00:55:39.660577+0000","last_became_peered":"2024-04-25T00:55:39.660577+0000","last_unstale":"2024-04-25T00:55:41.018376+0000","last_undegraded":"2024-04-25T00:55:41.018376+0000","last_fullsized":"2024-04-25T00:55:41.018376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.187193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637626+0000","last_change":"2024-04-25T00:55:38.610833+0000","last_active":"2024-04-25T00:55:40.637626+0000","last_peered":"2024-04-25T00:55:40.637626+0000","last_clean":"2024-04-25T00:55:40.637626+0000","last_became_active":"2024-04-25T00:55:38.610476+0000","last_became_peered":"2024-04-25T00:55:38.610476+0000","last_unstale":"2024-04-25T00:55:40.637626+0000","last_undegraded":"2024-04-25T00:55:40.637626+0000","last_fullsized":"2024-04-25T00:55:40.637626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638296+0000","last_change":"2024-04-25T00:55:39.643691+0000","last_active":"2024-04-25T00:55:40.638296+0000","last_peered":"2024-04-25T00:55:40.638296+0000","last_clean":"2024-04-25T00:55:40.638296+0000","last_became_active":"2024-04-25T00:55:39.643024+0000","last_became_peered":"2024-04-25T00:55:39.643024+0000","last_unstale":"2024-04-25T00:55:40.638296+0000","last_undegraded":"2024-04-25T00:55:40.638296+0000","last_fullsized":"2024-04-25T00:55:40.638296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:22.249474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020063+0000","last_change":"2024-04-25T00:55:38.623842+0000","last_active":"2024-04-25T00:55:41.020063+0000","last_peered":"2024-04-25T00:55:41.020063+0000","last_clean":"2024-04-25T00:55:41.020063+0000","last_became_active":"2024-04-25T00:55:38.623514+0000","last_became_peered":"2024-04-25T00:55:38.623514+0000","last_unstale":"2024-04-25T00:55:41.020063+0000","last_undegraded":"2024-04-25T00:55:41.020063+0000","last_fullsized":"2024-04-25T00:55:41.020063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017354+0000","last_change":"2024-04-25T00:55:39.658238+0000","last_active":"2024-04-25T00:55:41.017354+0000","last_peered":"2024-04-25T00:55:41.017354+0000","last_clean":"2024-04-25T00:55:41.017354+0000","last_became_active":"2024-04-25T00:55:39.657889+0000","last_became_peered":"2024-04-25T00:55:39.657889+0000","last_unstale":"2024-04-25T00:55:41.017354+0000","last_undegraded":"2024-04-25T00:55:41.017354+0000","last_fullsized":"2024-04-25T00:55:41.017354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016737+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:55:41.016737+0000","last_peered":"2024-04-25T00:55:41.016737+0000","last_clean":"2024-04-25T00:55:41.016737+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:55:41.016737+0000","last_undegraded":"2024-04-25T00:55:41.016737+0000","last_fullsized":"2024-04-25T00:55:41.016737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637451+0000","last_change":"2024-04-25T00:55:38.632700+0000","last_active":"2024-04-25T00:55:40.637451+0000","last_peered":"2024-04-25T00:55:40.637451+0000","last_clean":"2024-04-25T00:55:40.637451+0000","last_became_active":"2024-04-25T00:55:38.631901+0000","last_became_peered":"2024-04-25T00:55:38.631901+0000","last_unstale":"2024-04-25T00:55:40.637451+0000","last_undegraded":"2024-04-25T00:55:40.637451+0000","last_fullsized":"2024-04-25T00:55:40.637451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021271+0000","last_change":"2024-04-25T00:55:39.639013+0000","last_active":"2024-04-25T00:55:41.021271+0000","last_peered":"2024-04-25T00:55:41.021271+0000","last_clean":"2024-04-25T00:55:41.021271+0000","last_became_active":"2024-04-25T00:55:39.638189+0000","last_became_peered":"2024-04-25T00:55:39.638189+0000","last_unstale":"2024-04-25T00:55:41.021271+0000","last_undegraded":"2024-04-25T00:55:41.021271+0000","last_fullsized":"2024-04-25T00:55:41.021271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016724+0000","last_change":"2024-04-25T00:55:38.621598+0000","last_active":"2024-04-25T00:55:41.016724+0000","last_peered":"2024-04-25T00:55:41.016724+0000","last_clean":"2024-04-25T00:55:41.016724+0000","last_became_active":"2024-04-25T00:55:38.620844+0000","last_became_peered":"2024-04-25T00:55:38.620844+0000","last_unstale":"2024-04-25T00:55:41.016724+0000","last_undegraded":"2024-04-25T00:55:41.016724+0000","last_fullsized":"2024-04-25T00:55:41.016724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019793+0000","last_change":"2024-04-25T00:55:39.626650+0000","last_active":"2024-04-25T00:55:41.019793+0000","last_peered":"2024-04-25T00:55:41.019793+0000","last_clean":"2024-04-25T00:55:41.019793+0000","last_became_active":"2024-04-25T00:55:39.625876+0000","last_became_peered":"2024-04-25T00:55:39.625876+0000","last_unstale":"2024-04-25T00:55:41.019793+0000","last_undegraded":"2024-04-25T00:55:41.019793+0000","last_fullsized":"2024-04-25T00:55:41.019793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018447+0000","last_change":"2024-04-25T00:55:38.605319+0000","last_active":"2024-04-25T00:55:41.018447+0000","last_peered":"2024-04-25T00:55:41.018447+0000","last_clean":"2024-04-25T00:55:41.018447+0000","last_became_active":"2024-04-25T00:55:38.604920+0000","last_became_peered":"2024-04-25T00:55:38.604920+0000","last_unstale":"2024-04-25T00:55:41.018447+0000","last_undegraded":"2024-04-25T00:55:41.018447+0000","last_fullsized":"2024-04-25T00:55:41.018447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636431+0000","last_change":"2024-04-25T00:55:39.635799+0000","last_active":"2024-04-25T00:55:40.636431+0000","last_peered":"2024-04-25T00:55:40.636431+0000","last_clean":"2024-04-25T00:55:40.636431+0000","last_became_active":"2024-04-25T00:55:39.633616+0000","last_became_peered":"2024-04-25T00:55:39.633616+0000","last_unstale":"2024-04-25T00:55:40.636431+0000","last_undegraded":"2024-04-25T00:55:40.636431+0000","last_fullsized":"2024-04-25T00:55:40.636431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638712+0000","last_change":"2024-04-25T00:55:38.638178+0000","last_active":"2024-04-25T00:55:40.638712+0000","last_peered":"2024-04-25T00:55:40.638712+0000","last_clean":"2024-04-25T00:55:40.638712+0000","last_became_active":"2024-04-25T00:55:38.637419+0000","last_became_peered":"2024-04-25T00:55:38.637419+0000","last_unstale":"2024-04-25T00:55:40.638712+0000","last_undegraded":"2024-04-25T00:55:40.638712+0000","last_fullsized":"2024-04-25T00:55:40.638712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636532+0000","last_change":"2024-04-25T00:55:39.635022+0000","last_active":"2024-04-25T00:55:40.636532+0000","last_peered":"2024-04-25T00:55:40.636532+0000","last_clean":"2024-04-25T00:55:40.636532+0000","last_became_active":"2024-04-25T00:55:39.634107+0000","last_became_peered":"2024-04-25T00:55:39.634107+0000","last_unstale":"2024-04-25T00:55:40.636532+0000","last_undegraded":"2024-04-25T00:55:40.636532+0000","last_fullsized":"2024-04-25T00:55:40.636532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.087648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637039+0000","last_change":"2024-04-25T00:55:38.607260+0000","last_active":"2024-04-25T00:55:40.637039+0000","last_peered":"2024-04-25T00:55:40.637039+0000","last_clean":"2024-04-25T00:55:40.637039+0000","last_became_active":"2024-04-25T00:55:38.606910+0000","last_became_peered":"2024-04-25T00:55:38.606910+0000","last_unstale":"2024-04-25T00:55:40.637039+0000","last_undegraded":"2024-04-25T00:55:40.637039+0000","last_fullsized":"2024-04-25T00:55:40.637039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:25.842602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637823+0000","last_change":"2024-04-25T00:55:39.641228+0000","last_active":"2024-04-25T00:55:40.637823+0000","last_peered":"2024-04-25T00:55:40.637823+0000","last_clean":"2024-04-25T00:55:40.637823+0000","last_became_active":"2024-04-25T00:55:39.640434+0000","last_became_peered":"2024-04-25T00:55:39.640434+0000","last_unstale":"2024-04-25T00:55:40.637823+0000","last_undegraded":"2024-04-25T00:55:40.637823+0000","last_fullsized":"2024-04-25T00:55:40.637823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.552805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018046+0000","last_change":"2024-04-25T00:55:38.630316+0000","last_active":"2024-04-25T00:55:41.018046+0000","last_peered":"2024-04-25T00:55:41.018046+0000","last_clean":"2024-04-25T00:55:41.018046+0000","last_became_active":"2024-04-25T00:55:38.629417+0000","last_became_peered":"2024-04-25T00:55:38.629417+0000","last_unstale":"2024-04-25T00:55:41.018046+0000","last_undegraded":"2024-04-25T00:55:41.018046+0000","last_fullsized":"2024-04-25T00:55:41.018046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636661+0000","last_change":"2024-04-25T00:55:39.633030+0000","last_active":"2024-04-25T00:55:40.636661+0000","last_peered":"2024-04-25T00:55:40.636661+0000","last_clean":"2024-04-25T00:55:40.636661+0000","last_became_active":"2024-04-25T00:55:39.632300+0000","last_became_peered":"2024-04-25T00:55:39.632300+0000","last_unstale":"2024-04-25T00:55:40.636661+0000","last_undegraded":"2024-04-25T00:55:40.636661+0000","last_fullsized":"2024-04-25T00:55:40.636661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652457+0000","last_change":"2024-04-25T00:55:38.634050+0000","last_active":"2024-04-25T00:55:40.652457+0000","last_peered":"2024-04-25T00:55:40.652457+0000","last_clean":"2024-04-25T00:55:40.652457+0000","last_became_active":"2024-04-25T00:55:38.633672+0000","last_became_peered":"2024-04-25T00:55:38.633672+0000","last_unstale":"2024-04-25T00:55:40.652457+0000","last_undegraded":"2024-04-25T00:55:40.652457+0000","last_fullsized":"2024-04-25T00:55:40.652457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.022010+0000","last_change":"2024-04-25T00:55:39.652710+0000","last_active":"2024-04-25T00:55:41.022010+0000","last_peered":"2024-04-25T00:55:41.022010+0000","last_clean":"2024-04-25T00:55:41.022010+0000","last_became_active":"2024-04-25T00:55:39.652302+0000","last_became_peered":"2024-04-25T00:55:39.652302+0000","last_unstale":"2024-04-25T00:55:41.022010+0000","last_undegraded":"2024-04-25T00:55:41.022010+0000","last_fullsized":"2024-04-25T00:55:41.022010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652764+0000","last_change":"2024-04-25T00:55:38.636274+0000","last_active":"2024-04-25T00:55:40.652764+0000","last_peered":"2024-04-25T00:55:40.652764+0000","last_clean":"2024-04-25T00:55:40.652764+0000","last_became_active":"2024-04-25T00:55:38.635626+0000","last_became_peered":"2024-04-25T00:55:38.635626+0000","last_unstale":"2024-04-25T00:55:40.652764+0000","last_undegraded":"2024-04-25T00:55:40.652764+0000","last_fullsized":"2024-04-25T00:55:40.652764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018903+0000","last_change":"2024-04-25T00:55:39.635041+0000","last_active":"2024-04-25T00:55:41.018903+0000","last_peered":"2024-04-25T00:55:41.018903+0000","last_clean":"2024-04-25T00:55:41.018903+0000","last_became_active":"2024-04-25T00:55:39.634758+0000","last_became_peered":"2024-04-25T00:55:39.634758+0000","last_unstale":"2024-04-25T00:55:41.018903+0000","last_undegraded":"2024-04-25T00:55:41.018903+0000","last_fullsized":"2024-04-25T00:55:41.018903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:53.762656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016600+0000","last_change":"2024-04-25T00:55:38.630288+0000","last_active":"2024-04-25T00:55:41.016600+0000","last_peered":"2024-04-25T00:55:41.016600+0000","last_clean":"2024-04-25T00:55:41.016600+0000","last_became_active":"2024-04-25T00:55:38.629355+0000","last_became_peered":"2024-04-25T00:55:38.629355+0000","last_unstale":"2024-04-25T00:55:41.016600+0000","last_undegraded":"2024-04-25T00:55:41.016600+0000","last_fullsized":"2024-04-25T00:55:41.016600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636226+0000","last_change":"2024-04-25T00:55:39.623782+0000","last_active":"2024-04-25T00:55:40.636226+0000","last_peered":"2024-04-25T00:55:40.636226+0000","last_clean":"2024-04-25T00:55:40.636226+0000","last_became_active":"2024-04-25T00:55:39.622815+0000","last_became_peered":"2024-04-25T00:55:39.622815+0000","last_unstale":"2024-04-25T00:55:40.636226+0000","last_undegraded":"2024-04-25T00:55:40.636226+0000","last_fullsized":"2024-04-25T00:55:40.636226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:03.896351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637109+0000","last_change":"2024-04-25T00:55:38.618497+0000","last_active":"2024-04-25T00:55:40.637109+0000","last_peered":"2024-04-25T00:55:40.637109+0000","last_clean":"2024-04-25T00:55:40.637109+0000","last_became_active":"2024-04-25T00:55:38.618150+0000","last_became_peered":"2024-04-25T00:55:38.618150+0000","last_unstale":"2024-04-25T00:55:40.637109+0000","last_undegraded":"2024-04-25T00:55:40.637109+0000","last_fullsized":"2024-04-25T00:55:40.637109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021086+0000","last_change":"2024-04-25T00:55:39.640734+0000","last_active":"2024-04-25T00:55:41.021086+0000","last_peered":"2024-04-25T00:55:41.021086+0000","last_clean":"2024-04-25T00:55:41.021086+0000","last_became_active":"2024-04-25T00:55:39.639372+0000","last_became_peered":"2024-04-25T00:55:39.639372+0000","last_unstale":"2024-04-25T00:55:41.021086+0000","last_undegraded":"2024-04-25T00:55:41.021086+0000","last_fullsized":"2024-04-25T00:55:41.021086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636950+0000","last_change":"2024-04-25T00:55:38.613079+0000","last_active":"2024-04-25T00:55:40.636950+0000","last_peered":"2024-04-25T00:55:40.636950+0000","last_clean":"2024-04-25T00:55:40.636950+0000","last_became_active":"2024-04-25T00:55:38.612643+0000","last_became_peered":"2024-04-25T00:55:38.612643+0000","last_unstale":"2024-04-25T00:55:40.636950+0000","last_undegraded":"2024-04-25T00:55:40.636950+0000","last_fullsized":"2024-04-25T00:55:40.636950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021795+0000","last_change":"2024-04-25T00:55:39.640875+0000","last_active":"2024-04-25T00:55:41.021795+0000","last_peered":"2024-04-25T00:55:41.021795+0000","last_clean":"2024-04-25T00:55:41.021795+0000","last_became_active":"2024-04-25T00:55:39.638658+0000","last_became_peered":"2024-04-25T00:55:39.638658+0000","last_unstale":"2024-04-25T00:55:41.021795+0000","last_undegraded":"2024-04-25T00:55:41.021795+0000","last_fullsized":"2024-04-25T00:55:41.021795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017957+0000","last_change":"2024-04-25T00:55:38.624428+0000","last_active":"2024-04-25T00:55:41.017957+0000","last_peered":"2024-04-25T00:55:41.017957+0000","last_clean":"2024-04-25T00:55:41.017957+0000","last_became_active":"2024-04-25T00:55:38.622736+0000","last_became_peered":"2024-04-25T00:55:38.622736+0000","last_unstale":"2024-04-25T00:55:41.017957+0000","last_undegraded":"2024-04-25T00:55:41.017957+0000","last_fullsized":"2024-04-25T00:55:41.017957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:26.077118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652902+0000","last_change":"2024-04-25T00:55:39.631349+0000","last_active":"2024-04-25T00:55:40.652902+0000","last_peered":"2024-04-25T00:55:40.652902+0000","last_clean":"2024-04-25T00:55:40.652902+0000","last_became_active":"2024-04-25T00:55:39.630507+0000","last_became_peered":"2024-04-25T00:55:39.630507+0000","last_unstale":"2024-04-25T00:55:40.652902+0000","last_undegraded":"2024-04-25T00:55:40.652902+0000","last_fullsized":"2024-04-25T00:55:40.652902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637806+0000","last_change":"2024-04-25T00:55:39.628089+0000","last_active":"2024-04-25T00:55:40.637806+0000","last_peered":"2024-04-25T00:55:40.637806+0000","last_clean":"2024-04-25T00:55:40.637806+0000","last_became_active":"2024-04-25T00:55:39.627070+0000","last_became_peered":"2024-04-25T00:55:39.627070+0000","last_unstale":"2024-04-25T00:55:40.637806+0000","last_undegraded":"2024-04-25T00:55:40.637806+0000","last_fullsized":"2024-04-25T00:55:40.637806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017214+0000","last_change":"2024-04-25T00:55:38.634723+0000","last_active":"2024-04-25T00:55:41.017214+0000","last_peered":"2024-04-25T00:55:41.017214+0000","last_clean":"2024-04-25T00:55:41.017214+0000","last_became_active":"2024-04-25T00:55:38.633867+0000","last_became_peered":"2024-04-25T00:55:38.633867+0000","last_unstale":"2024-04-25T00:55:41.017214+0000","last_undegraded":"2024-04-25T00:55:41.017214+0000","last_fullsized":"2024-04-25T00:55:41.017214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638050+0000","last_change":"2024-04-25T00:55:39.618027+0000","last_active":"2024-04-25T00:55:40.638050+0000","last_peered":"2024-04-25T00:55:40.638050+0000","last_clean":"2024-04-25T00:55:40.638050+0000","last_became_active":"2024-04-25T00:55:39.617211+0000","last_became_peered":"2024-04-25T00:55:39.617211+0000","last_unstale":"2024-04-25T00:55:40.638050+0000","last_undegraded":"2024-04-25T00:55:40.638050+0000","last_fullsized":"2024-04-25T00:55:40.638050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:11.034367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017702+0000","last_change":"2024-04-25T00:55:38.636516+0000","last_active":"2024-04-25T00:55:41.017702+0000","last_peered":"2024-04-25T00:55:41.017702+0000","last_clean":"2024-04-25T00:55:41.017702+0000","last_became_active":"2024-04-25T00:55:38.635662+0000","last_became_peered":"2024-04-25T00:55:38.635662+0000","last_unstale":"2024-04-25T00:55:41.017702+0000","last_undegraded":"2024-04-25T00:55:41.017702+0000","last_fullsized":"2024-04-25T00:55:41.017702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018504+0000","last_change":"2024-04-25T00:55:39.622594+0000","last_active":"2024-04-25T00:55:41.018504+0000","last_peered":"2024-04-25T00:55:41.018504+0000","last_clean":"2024-04-25T00:55:41.018504+0000","last_became_active":"2024-04-25T00:55:39.621768+0000","last_became_peered":"2024-04-25T00:55:39.621768+0000","last_unstale":"2024-04-25T00:55:41.018504+0000","last_undegraded":"2024-04-25T00:55:41.018504+0000","last_fullsized":"2024-04-25T00:55:41.018504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:27.074899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021808+0000","last_change":"2024-04-25T00:55:38.609758+0000","last_active":"2024-04-25T00:55:41.021808+0000","last_peered":"2024-04-25T00:55:41.021808+0000","last_clean":"2024-04-25T00:55:41.021808+0000","last_became_active":"2024-04-25T00:55:38.609401+0000","last_became_peered":"2024-04-25T00:55:38.609401+0000","last_unstale":"2024-04-25T00:55:41.021808+0000","last_undegraded":"2024-04-25T00:55:41.021808+0000","last_fullsized":"2024-04-25T00:55:41.021808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:01.720059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021639+0000","last_change":"2024-04-25T00:55:39.657195+0000","last_active":"2024-04-25T00:55:41.021639+0000","last_peered":"2024-04-25T00:55:41.021639+0000","last_clean":"2024-04-25T00:55:41.021639+0000","last_became_active":"2024-04-25T00:55:39.656921+0000","last_became_peered":"2024-04-25T00:55:39.656921+0000","last_unstale":"2024-04-25T00:55:41.021639+0000","last_undegraded":"2024-04-25T00:55:41.021639+0000","last_fullsized":"2024-04-25T00:55:41.021639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021304+0000","last_change":"2024-04-25T00:55:38.629445+0000","last_active":"2024-04-25T00:55:41.021304+0000","last_peered":"2024-04-25T00:55:41.021304+0000","last_clean":"2024-04-25T00:55:41.021304+0000","last_became_active":"2024-04-25T00:55:38.628997+0000","last_became_peered":"2024-04-25T00:55:38.628997+0000","last_unstale":"2024-04-25T00:55:41.021304+0000","last_undegraded":"2024-04-25T00:55:41.021304+0000","last_fullsized":"2024-04-25T00:55:41.021304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:01.720059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:55:46.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:55:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.380+0000 7f3f61147640 1 Processor -- start 2024-04-25T00:55:46.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.380+0000 7f3f61147640 1 -- start start 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f5c0add70 0x7f3f5c145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c0aeec0 con 0x7f3f5c0b23f0 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c0af780 con 0x7f3f5c0b3ce0 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3f5c0b27d0 con 0x7f3f5c0add70 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f5a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3f5c0add70 0x7f3f5c145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f5ad9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3f5c0b23f0 0x7f3f5c072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57306/0 (socket says 172.21.15.112:57306) 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f5ad9d640 1 -- 172.21.15.112:0/2444954222 learned_addr learned my addr 172.21.15.112:0/2444954222 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f5a59c640 1 -- 172.21.15.112:0/2444954222 >> v1:172.21.15.187:6789/0 conn(0x7f3f5c0b3ce0 legacy=0x7f3f5c14fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f5a59c640 1 -- 172.21.15.112:0/2444954222 >> v1:172.21.15.112:6789/0 conn(0x7f3f5c0b23f0 legacy=0x7f3f5c072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.387 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f5a59c640 1 -- 172.21.15.112:0/2444954222 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c0afa20 con 0x7f3f5c0add70 2024-04-25T00:55:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f5a59c640 1 --2- 172.21.15.112:0/2444954222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c145200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0af950 tx=0x7f3f5001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e76bcc97107e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f59d9b640 1 -- 172.21.15.112:0/2444954222 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f500ab680 con 0x7f3f5c0add70 2024-04-25T00:55:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f59d9b640 1 -- 172.21.15.112:0/2444954222 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f500b0070 con 0x7f3f5c0add70 2024-04-25T00:55:46.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f59d9b640 1 -- 172.21.15.112:0/2444954222 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f500aba40 con 0x7f3f5c0add70 2024-04-25T00:55:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 -- 172.21.15.112:0/2444954222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 msgr2=0x7f3f5c145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 --2- 172.21.15.112:0/2444954222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c145200 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0af950 tx=0x7f3f5001c460 comp rx=0 tx=0).stop 2024-04-25T00:55:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 -- 172.21.15.112:0/2444954222 shutdown_connections 2024-04-25T00:55:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 --2- 172.21.15.112:0/2444954222 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c145200 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 -- 172.21.15.112:0/2444954222 >> 172.21.15.112:0/2444954222 conn(0x7f3f5c0b5980 msgr2=0x7f3f5c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 -- 172.21.15.112:0/2444954222 shutdown_connections 2024-04-25T00:55:46.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.384+0000 7f3f61147640 1 -- 172.21.15.112:0/2444954222 wait complete. 2024-04-25T00:55:46.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 Processor -- start 2024-04-25T00:55:46.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 -- start start 2024-04-25T00:55:46.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c137ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f5c0b23f0 0x7f3f5c1dffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f5c0b3ce0 0x7f3f5c1e0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0b52d0 con 0x7f3f5c0add70 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5c0b5150 con 0x7f3f5c0b3ce0 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3f5c153a80 con 0x7f3f5c0b23f0 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c137ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c137ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51380/0 (socket says 172.21.15.112:51380) 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 -- 172.21.15.112:0/858136261 learned_addr learned my addr 172.21.15.112:0/858136261 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 -- 172.21.15.112:0/858136261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f5c0b23f0 msgr2=0x7f3f5c1dffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f5c0b23f0 0x7f3f5c1dffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 -- 172.21.15.112:0/858136261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f5c0b3ce0 msgr2=0x7f3f5c1e0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f5c0b3ce0 0x7f3f5c1e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 -- 172.21.15.112:0/858136261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c1e39c0 con 0x7f3f5c0add70 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f5ad9d640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c137ee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400cb10 tx=0x7f3f5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f437fe640 1 -- 172.21.15.112:0/858136261 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54017070 con 0x7f3f5c0add70 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f437fe640 1 -- 172.21.15.112:0/858136261 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5400ac70 con 0x7f3f5c0add70 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c1e3ca0 con 0x7f3f5c0add70 2024-04-25T00:55:46.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f437fe640 1 -- 172.21.15.112:0/858136261 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54012620 con 0x7f3f5c0add70 2024-04-25T00:55:46.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f5c14fea0 con 0x7f3f5c0add70 2024-04-25T00:55:46.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5c04f580 con 0x7f3f5c0add70 2024-04-25T00:55:46.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f437fe640 1 -- 172.21.15.112:0/858136261 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3f5401a070 con 0x7f3f5c0add70 2024-04-25T00:55:46.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.388+0000 7f3f437fe640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3f480678c0 0x7f3f48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.392+0000 7f3f437fe640 1 -- 172.21.15.112:0/858136261 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3f540859f0 con 0x7f3f5c0add70 2024-04-25T00:55:46.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.392+0000 7f3f5a59c640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3f480678c0 0x7f3f48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.392+0000 7f3f5a59c640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3f480678c0 0x7f3f48069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f500af000 tx=0x7f3f50004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.396+0000 7f3f437fe640 1 -- 172.21.15.112:0/858136261 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3f540565c0 con 0x7f3f5c0add70 2024-04-25T00:55:46.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.536+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f5c1e41c0 con 0x7f3f480678c0 2024-04-25T00:55:46.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f437fe640 1 -- 172.21.15.112:0/858136261 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f3f500b0040 con 0x7f3f480678c0 2024-04-25T00:55:46.552 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:46.554 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3f480678c0 msgr2=0x7f3f48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3f480678c0 0x7f3f48069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3f500af000 tx=0x7f3f50004740 comp rx=0 tx=0).stop 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 msgr2=0x7f3f5c137ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c137ee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400cb10 tx=0x7f3f5400a060 comp rx=0 tx=0).stop 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 shutdown_connections 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3f480678c0 0x7f3f48069d80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3f5c0b3ce0 0x7f3f5c1e0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3f5c0b23f0 0x7f3f5c1dffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 --2- 172.21.15.112:0/858136261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3f5c0add70 0x7f3f5c137ee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.548+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 >> 172.21.15.112:0/858136261 conn(0x7f3f5c0b5980 msgr2=0x7f3f5c0b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:46.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.552+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 shutdown_connections 2024-04-25T00:55:46.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.552+0000 7f3f61147640 1 -- 172.21.15.112:0/858136261 wait complete. 2024-04-25T00:55:46.569 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T00:55:45.516289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218984,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755736,"statfs":{"total":773094113280,"available":772869873664,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.932808"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018125+0000","last_change":"2024-04-25T00:55:39.661241+0000","last_active":"2024-04-25T00:55:41.018125+0000","last_peered":"2024-04-25T00:55:41.018125+0000","last_clean":"2024-04-25T00:55:41.018125+0000","last_became_active":"2024-04-25T00:55:39.660150+0000","last_became_peered":"2024-04-25T00:55:39.660150+0000","last_unstale":"2024-04-25T00:55:41.018125+0000","last_undegraded":"2024-04-25T00:55:41.018125+0000","last_fullsized":"2024-04-25T00:55:41.018125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:11.034367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637050+0000","last_change":"2024-04-25T00:55:39.633699+0000","last_active":"2024-04-25T00:55:40.637050+0000","last_peered":"2024-04-25T00:55:40.637050+0000","last_clean":"2024-04-25T00:55:40.637050+0000","last_became_active":"2024-04-25T00:55:39.632942+0000","last_became_peered":"2024-04-25T00:55:39.632942+0000","last_unstale":"2024-04-25T00:55:40.637050+0000","last_undegraded":"2024-04-25T00:55:40.637050+0000","last_fullsized":"2024-04-25T00:55:40.637050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018670+0000","last_change":"2024-04-25T00:55:39.629762+0000","last_active":"2024-04-25T00:55:41.018670+0000","last_peered":"2024-04-25T00:55:41.018670+0000","last_clean":"2024-04-25T00:55:41.018670+0000","last_became_active":"2024-04-25T00:55:39.628798+0000","last_became_peered":"2024-04-25T00:55:39.628798+0000","last_unstale":"2024-04-25T00:55:41.018670+0000","last_undegraded":"2024-04-25T00:55:41.018670+0000","last_fullsized":"2024-04-25T00:55:41.018670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:24:23.939072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637845+0000","last_change":"2024-04-25T00:55:39.643398+0000","last_active":"2024-04-25T00:55:40.637845+0000","last_peered":"2024-04-25T00:55:40.637845+0000","last_clean":"2024-04-25T00:55:40.637845+0000","last_became_active":"2024-04-25T00:55:39.642844+0000","last_became_peered":"2024-04-25T00:55:39.642844+0000","last_unstale":"2024-04-25T00:55:40.637845+0000","last_undegraded":"2024-04-25T00:55:40.637845+0000","last_fullsized":"2024-04-25T00:55:40.637845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017483+0000","last_change":"2024-04-25T00:55:39.649899+0000","last_active":"2024-04-25T00:55:41.017483+0000","last_peered":"2024-04-25T00:55:41.017483+0000","last_clean":"2024-04-25T00:55:41.017483+0000","last_became_active":"2024-04-25T00:55:39.648852+0000","last_became_peered":"2024-04-25T00:55:39.648852+0000","last_unstale":"2024-04-25T00:55:41.017483+0000","last_undegraded":"2024-04-25T00:55:41.017483+0000","last_fullsized":"2024-04-25T00:55:41.017483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638862+0000","last_change":"2024-04-25T00:55:39.635369+0000","last_active":"2024-04-25T00:55:40.638862+0000","last_peered":"2024-04-25T00:55:40.638862+0000","last_clean":"2024-04-25T00:55:40.638862+0000","last_became_active":"2024-04-25T00:55:39.634678+0000","last_became_peered":"2024-04-25T00:55:39.634678+0000","last_unstale":"2024-04-25T00:55:40.638862+0000","last_undegraded":"2024-04-25T00:55:40.638862+0000","last_fullsized":"2024-04-25T00:55:40.638862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:30.556421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016962+0000","last_change":"2024-04-25T00:55:39.646309+0000","last_active":"2024-04-25T00:55:41.016962+0000","last_peered":"2024-04-25T00:55:41.016962+0000","last_clean":"2024-04-25T00:55:41.016962+0000","last_became_active":"2024-04-25T00:55:39.645778+0000","last_became_peered":"2024-04-25T00:55:39.645778+0000","last_unstale":"2024-04-25T00:55:41.016962+0000","last_undegraded":"2024-04-25T00:55:41.016962+0000","last_fullsized":"2024-04-25T00:55:41.016962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020590+0000","last_change":"2024-04-25T00:55:39.643250+0000","last_active":"2024-04-25T00:55:41.020590+0000","last_peered":"2024-04-25T00:55:41.020590+0000","last_clean":"2024-04-25T00:55:41.020590+0000","last_became_active":"2024-04-25T00:55:39.642948+0000","last_became_peered":"2024-04-25T00:55:39.642948+0000","last_unstale":"2024-04-25T00:55:41.020590+0000","last_undegraded":"2024-04-25T00:55:41.020590+0000","last_fullsized":"2024-04-25T00:55:41.020590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:53.762656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018212+0000","last_change":"2024-04-25T00:55:39.658665+0000","last_active":"2024-04-25T00:55:41.018212+0000","last_peered":"2024-04-25T00:55:41.018212+0000","last_clean":"2024-04-25T00:55:41.018212+0000","last_became_active":"2024-04-25T00:55:39.658276+0000","last_became_peered":"2024-04-25T00:55:39.658276+0000","last_unstale":"2024-04-25T00:55:41.018212+0000","last_undegraded":"2024-04-25T00:55:41.018212+0000","last_fullsized":"2024-04-25T00:55:41.018212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:47.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652697+0000","last_change":"2024-04-25T00:55:39.660203+0000","last_active":"2024-04-25T00:55:40.652697+0000","last_peered":"2024-04-25T00:55:40.652697+0000","last_clean":"2024-04-25T00:55:40.652697+0000","last_became_active":"2024-04-25T00:55:39.656617+0000","last_became_peered":"2024-04-25T00:55:39.656617+0000","last_unstale":"2024-04-25T00:55:40.652697+0000","last_undegraded":"2024-04-25T00:55:40.652697+0000","last_fullsized":"2024-04-25T00:55:40.652697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652875+0000","last_change":"2024-04-25T00:55:39.638495+0000","last_active":"2024-04-25T00:55:40.652875+0000","last_peered":"2024-04-25T00:55:40.652875+0000","last_clean":"2024-04-25T00:55:40.652875+0000","last_became_active":"2024-04-25T00:55:39.631516+0000","last_became_peered":"2024-04-25T00:55:39.631516+0000","last_unstale":"2024-04-25T00:55:40.652875+0000","last_undegraded":"2024-04-25T00:55:40.652875+0000","last_fullsized":"2024-04-25T00:55:40.652875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636317+0000","last_change":"2024-04-25T00:55:39.657591+0000","last_active":"2024-04-25T00:55:40.636317+0000","last_peered":"2024-04-25T00:55:40.636317+0000","last_clean":"2024-04-25T00:55:40.636317+0000","last_became_active":"2024-04-25T00:55:39.657259+0000","last_became_peered":"2024-04-25T00:55:39.657259+0000","last_unstale":"2024-04-25T00:55:40.636317+0000","last_undegraded":"2024-04-25T00:55:40.636317+0000","last_fullsized":"2024-04-25T00:55:40.636317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:56.786582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.022200+0000","last_change":"2024-04-25T00:55:39.656495+0000","last_active":"2024-04-25T00:55:41.022200+0000","last_peered":"2024-04-25T00:55:41.022200+0000","last_clean":"2024-04-25T00:55:41.022200+0000","last_became_active":"2024-04-25T00:55:39.655994+0000","last_became_peered":"2024-04-25T00:55:39.655994+0000","last_unstale":"2024-04-25T00:55:41.022200+0000","last_undegraded":"2024-04-25T00:55:41.022200+0000","last_fullsized":"2024-04-25T00:55:41.022200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:58.389264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019282+0000","last_change":"2024-04-25T00:55:39.636352+0000","last_active":"2024-04-25T00:55:41.019282+0000","last_peered":"2024-04-25T00:55:41.019282+0000","last_clean":"2024-04-25T00:55:41.019282+0000","last_became_active":"2024-04-25T00:55:39.636013+0000","last_became_peered":"2024-04-25T00:55:39.636013+0000","last_unstale":"2024-04-25T00:55:41.019282+0000","last_undegraded":"2024-04-25T00:55:41.019282+0000","last_fullsized":"2024-04-25T00:55:41.019282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636741+0000","last_change":"2024-04-25T00:55:39.634943+0000","last_active":"2024-04-25T00:55:40.636741+0000","last_peered":"2024-04-25T00:55:40.636741+0000","last_clean":"2024-04-25T00:55:40.636741+0000","last_became_active":"2024-04-25T00:55:39.634116+0000","last_became_peered":"2024-04-25T00:55:39.634116+0000","last_unstale":"2024-04-25T00:55:40.636741+0000","last_undegraded":"2024-04-25T00:55:40.636741+0000","last_fullsized":"2024-04-25T00:55:40.636741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637210+0000","last_change":"2024-04-25T00:55:39.630149+0000","last_active":"2024-04-25T00:55:40.637210+0000","last_peered":"2024-04-25T00:55:40.637210+0000","last_clean":"2024-04-25T00:55:40.637210+0000","last_became_active":"2024-04-25T00:55:39.629387+0000","last_became_peered":"2024-04-25T00:55:39.629387+0000","last_unstale":"2024-04-25T00:55:40.637210+0000","last_undegraded":"2024-04-25T00:55:40.637210+0000","last_fullsized":"2024-04-25T00:55:40.637210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638541+0000","last_change":"2024-04-25T00:55:39.650995+0000","last_active":"2024-04-25T00:55:40.638541+0000","last_peered":"2024-04-25T00:55:40.638541+0000","last_clean":"2024-04-25T00:55:40.638541+0000","last_became_active":"2024-04-25T00:55:39.650711+0000","last_became_peered":"2024-04-25T00:55:39.650711+0000","last_unstale":"2024-04-25T00:55:40.638541+0000","last_undegraded":"2024-04-25T00:55:40.638541+0000","last_fullsized":"2024-04-25T00:55:40.638541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.187193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019011+0000","last_change":"2024-04-25T00:55:39.656754+0000","last_active":"2024-04-25T00:55:41.019011+0000","last_peered":"2024-04-25T00:55:41.019011+0000","last_clean":"2024-04-25T00:55:41.019011+0000","last_became_active":"2024-04-25T00:55:39.656397+0000","last_became_peered":"2024-04-25T00:55:39.656397+0000","last_unstale":"2024-04-25T00:55:41.019011+0000","last_undegraded":"2024-04-25T00:55:41.019011+0000","last_fullsized":"2024-04-25T00:55:41.019011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.087648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018205+0000","last_change":"2024-04-25T00:55:39.648560+0000","last_active":"2024-04-25T00:55:41.018205+0000","last_peered":"2024-04-25T00:55:41.018205+0000","last_clean":"2024-04-25T00:55:41.018205+0000","last_became_active":"2024-04-25T00:55:39.647080+0000","last_became_peered":"2024-04-25T00:55:39.647080+0000","last_unstale":"2024-04-25T00:55:41.018205+0000","last_undegraded":"2024-04-25T00:55:41.018205+0000","last_fullsized":"2024-04-25T00:55:41.018205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:22.249474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.653020+0000","last_change":"2024-04-25T00:55:39.659058+0000","last_active":"2024-04-25T00:55:40.653020+0000","last_peered":"2024-04-25T00:55:40.653020+0000","last_clean":"2024-04-25T00:55:40.653020+0000","last_became_active":"2024-04-25T00:55:39.657152+0000","last_became_peered":"2024-04-25T00:55:39.657152+0000","last_unstale":"2024-04-25T00:55:40.653020+0000","last_undegraded":"2024-04-25T00:55:40.653020+0000","last_fullsized":"2024-04-25T00:55:40.653020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020095+0000","last_change":"2024-04-25T00:55:39.650890+0000","last_active":"2024-04-25T00:55:41.020095+0000","last_peered":"2024-04-25T00:55:41.020095+0000","last_clean":"2024-04-25T00:55:41.020095+0000","last_became_active":"2024-04-25T00:55:39.650235+0000","last_became_peered":"2024-04-25T00:55:39.650235+0000","last_unstale":"2024-04-25T00:55:41.020095+0000","last_undegraded":"2024-04-25T00:55:41.020095+0000","last_fullsized":"2024-04-25T00:55:41.020095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638004+0000","last_change":"2024-04-25T00:55:39.646725+0000","last_active":"2024-04-25T00:55:40.638004+0000","last_peered":"2024-04-25T00:55:40.638004+0000","last_clean":"2024-04-25T00:55:40.638004+0000","last_became_active":"2024-04-25T00:55:39.646192+0000","last_became_peered":"2024-04-25T00:55:39.646192+0000","last_unstale":"2024-04-25T00:55:40.638004+0000","last_undegraded":"2024-04-25T00:55:40.638004+0000","last_fullsized":"2024-04-25T00:55:40.638004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:51.008333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020594+0000","last_change":"2024-04-25T00:55:39.642303+0000","last_active":"2024-04-25T00:55:41.020594+0000","last_peered":"2024-04-25T00:55:41.020594+0000","last_clean":"2024-04-25T00:55:41.020594+0000","last_became_active":"2024-04-25T00:55:39.641627+0000","last_became_peered":"2024-04-25T00:55:39.641627+0000","last_unstale":"2024-04-25T00:55:41.020594+0000","last_undegraded":"2024-04-25T00:55:41.020594+0000","last_fullsized":"2024-04-25T00:55:41.020594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637271+0000","last_change":"2024-04-25T00:55:39.643919+0000","last_active":"2024-04-25T00:55:40.637271+0000","last_peered":"2024-04-25T00:55:40.637271+0000","last_clean":"2024-04-25T00:55:40.637271+0000","last_became_active":"2024-04-25T00:55:39.643196+0000","last_became_peered":"2024-04-25T00:55:39.643196+0000","last_unstale":"2024-04-25T00:55:40.637271+0000","last_undegraded":"2024-04-25T00:55:40.637271+0000","last_fullsized":"2024-04-25T00:55:40.637271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021581+0000","last_change":"2024-04-25T00:55:39.641462+0000","last_active":"2024-04-25T00:55:41.021581+0000","last_peered":"2024-04-25T00:55:41.021581+0000","last_clean":"2024-04-25T00:55:41.021581+0000","last_became_active":"2024-04-25T00:55:39.640648+0000","last_became_peered":"2024-04-25T00:55:39.640648+0000","last_unstale":"2024-04-25T00:55:41.021581+0000","last_undegraded":"2024-04-25T00:55:41.021581+0000","last_fullsized":"2024-04-25T00:55:41.021581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652767+0000","last_change":"2024-04-25T00:55:39.648816+0000","last_active":"2024-04-25T00:55:40.652767+0000","last_peered":"2024-04-25T00:55:40.652767+0000","last_clean":"2024-04-25T00:55:40.652767+0000","last_became_active":"2024-04-25T00:55:39.648337+0000","last_became_peered":"2024-04-25T00:55:39.648337+0000","last_unstale":"2024-04-25T00:55:40.652767+0000","last_undegraded":"2024-04-25T00:55:40.652767+0000","last_fullsized":"2024-04-25T00:55:40.652767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637430+0000","last_change":"2024-04-25T00:55:39.644333+0000","last_active":"2024-04-25T00:55:40.637430+0000","last_peered":"2024-04-25T00:55:40.637430+0000","last_clean":"2024-04-25T00:55:40.637430+0000","last_became_active":"2024-04-25T00:55:39.643706+0000","last_became_peered":"2024-04-25T00:55:39.643706+0000","last_unstale":"2024-04-25T00:55:40.637430+0000","last_undegraded":"2024-04-25T00:55:40.637430+0000","last_fullsized":"2024-04-25T00:55:40.637430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021853+0000","last_change":"2024-04-25T00:55:39.631560+0000","last_active":"2024-04-25T00:55:41.021853+0000","last_peered":"2024-04-25T00:55:41.021853+0000","last_clean":"2024-04-25T00:55:41.021853+0000","last_became_active":"2024-04-25T00:55:39.630796+0000","last_became_peered":"2024-04-25T00:55:39.630796+0000","last_unstale":"2024-04-25T00:55:41.021853+0000","last_undegraded":"2024-04-25T00:55:41.021853+0000","last_fullsized":"2024-04-25T00:55:41.021853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:19:30.518598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020172+0000","last_change":"2024-04-25T00:55:39.643191+0000","last_active":"2024-04-25T00:55:41.020172+0000","last_peered":"2024-04-25T00:55:41.020172+0000","last_clean":"2024-04-25T00:55:41.020172+0000","last_became_active":"2024-04-25T00:55:39.642846+0000","last_became_peered":"2024-04-25T00:55:39.642846+0000","last_unstale":"2024-04-25T00:55:41.020172+0000","last_undegraded":"2024-04-25T00:55:41.020172+0000","last_fullsized":"2024-04-25T00:55:41.020172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.103122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638416+0000","last_change":"2024-04-25T00:55:39.643174+0000","last_active":"2024-04-25T00:55:40.638416+0000","last_peered":"2024-04-25T00:55:40.638416+0000","last_clean":"2024-04-25T00:55:40.638416+0000","last_became_active":"2024-04-25T00:55:39.642575+0000","last_became_peered":"2024-04-25T00:55:39.642575+0000","last_unstale":"2024-04-25T00:55:40.638416+0000","last_undegraded":"2024-04-25T00:55:40.638416+0000","last_fullsized":"2024-04-25T00:55:40.638416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:47.929873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018168+0000","last_change":"2024-04-25T00:55:39.627384+0000","last_active":"2024-04-25T00:55:41.018168+0000","last_peered":"2024-04-25T00:55:41.018168+0000","last_clean":"2024-04-25T00:55:41.018168+0000","last_became_active":"2024-04-25T00:55:39.627078+0000","last_became_peered":"2024-04-25T00:55:39.627078+0000","last_unstale":"2024-04-25T00:55:41.018168+0000","last_undegraded":"2024-04-25T00:55:41.018168+0000","last_fullsized":"2024-04-25T00:55:41.018168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:21.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652573+0000","last_change":"2024-04-25T00:55:39.655121+0000","last_active":"2024-04-25T00:55:40.652573+0000","last_peered":"2024-04-25T00:55:40.652573+0000","last_clean":"2024-04-25T00:55:40.652573+0000","last_became_active":"2024-04-25T00:55:39.654774+0000","last_became_peered":"2024-04-25T00:55:39.654774+0000","last_unstale":"2024-04-25T00:55:40.652573+0000","last_undegraded":"2024-04-25T00:55:40.652573+0000","last_fullsized":"2024-04-25T00:55:40.652573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636316+0000","last_change":"2024-04-25T00:55:39.634737+0000","last_active":"2024-04-25T00:55:40.636316+0000","last_peered":"2024-04-25T00:55:40.636316+0000","last_clean":"2024-04-25T00:55:40.636316+0000","last_became_active":"2024-04-25T00:55:39.633943+0000","last_became_peered":"2024-04-25T00:55:39.633943+0000","last_unstale":"2024-04-25T00:55:40.636316+0000","last_undegraded":"2024-04-25T00:55:40.636316+0000","last_fullsized":"2024-04-25T00:55:40.636316+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:09.262927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021945+0000","last_change":"2024-04-25T00:55:38.629602+0000","last_active":"2024-04-25T00:55:41.021945+0000","last_peered":"2024-04-25T00:55:41.021945+0000","last_clean":"2024-04-25T00:55:41.021945+0000","last_became_active":"2024-04-25T00:55:38.628956+0000","last_became_peered":"2024-04-25T00:55:38.628956+0000","last_unstale":"2024-04-25T00:55:41.021945+0000","last_undegraded":"2024-04-25T00:55:41.021945+0000","last_fullsized":"2024-04-25T00:55:41.021945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:50.369672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020489+0000","last_change":"2024-04-25T00:55:39.630640+0000","last_active":"2024-04-25T00:55:41.020489+0000","last_peered":"2024-04-25T00:55:41.020489+0000","last_clean":"2024-04-25T00:55:41.020489+0000","last_became_active":"2024-04-25T00:55:39.629906+0000","last_became_peered":"2024-04-25T00:55:39.629906+0000","last_unstale":"2024-04-25T00:55:41.020489+0000","last_undegraded":"2024-04-25T00:55:41.020489+0000","last_fullsized":"2024-04-25T00:55:41.020489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652654+0000","last_change":"2024-04-25T00:55:38.626215+0000","last_active":"2024-04-25T00:55:40.652654+0000","last_peered":"2024-04-25T00:55:40.652654+0000","last_clean":"2024-04-25T00:55:40.652654+0000","last_became_active":"2024-04-25T00:55:38.625439+0000","last_became_peered":"2024-04-25T00:55:38.625439+0000","last_unstale":"2024-04-25T00:55:40.652654+0000","last_undegraded":"2024-04-25T00:55:40.652654+0000","last_fullsized":"2024-04-25T00:55:40.652654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020655+0000","last_change":"2024-04-25T00:55:38.626401+0000","last_active":"2024-04-25T00:55:41.020655+0000","last_peered":"2024-04-25T00:55:41.020655+0000","last_clean":"2024-04-25T00:55:41.020655+0000","last_became_active":"2024-04-25T00:55:38.625884+0000","last_became_peered":"2024-04-25T00:55:38.625884+0000","last_unstale":"2024-04-25T00:55:41.020655+0000","last_undegraded":"2024-04-25T00:55:41.020655+0000","last_fullsized":"2024-04-25T00:55:41.020655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020981+0000","last_change":"2024-04-25T00:55:39.638181+0000","last_active":"2024-04-25T00:55:41.020981+0000","last_peered":"2024-04-25T00:55:41.020981+0000","last_clean":"2024-04-25T00:55:41.020981+0000","last_became_active":"2024-04-25T00:55:39.637900+0000","last_became_peered":"2024-04-25T00:55:39.637900+0000","last_unstale":"2024-04-25T00:55:41.020981+0000","last_undegraded":"2024-04-25T00:55:41.020981+0000","last_fullsized":"2024-04-25T00:55:41.020981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637316+0000","last_change":"2024-04-25T00:55:38.621335+0000","last_active":"2024-04-25T00:55:40.637316+0000","last_peered":"2024-04-25T00:55:40.637316+0000","last_clean":"2024-04-25T00:55:40.637316+0000","last_became_active":"2024-04-25T00:55:38.620467+0000","last_became_peered":"2024-04-25T00:55:38.620467+0000","last_unstale":"2024-04-25T00:55:40.637316+0000","last_undegraded":"2024-04-25T00:55:40.637316+0000","last_fullsized":"2024-04-25T00:55:40.637316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636806+0000","last_change":"2024-04-25T00:55:39.642780+0000","last_active":"2024-04-25T00:55:40.636806+0000","last_peered":"2024-04-25T00:55:40.636806+0000","last_clean":"2024-04-25T00:55:40.636806+0000","last_became_active":"2024-04-25T00:55:39.642410+0000","last_became_peered":"2024-04-25T00:55:39.642410+0000","last_unstale":"2024-04-25T00:55:40.636806+0000","last_undegraded":"2024-04-25T00:55:40.636806+0000","last_fullsized":"2024-04-25T00:55:40.636806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652622+0000","last_change":"2024-04-25T00:55:38.634413+0000","last_active":"2024-04-25T00:55:40.652622+0000","last_peered":"2024-04-25T00:55:40.652622+0000","last_clean":"2024-04-25T00:55:40.652622+0000","last_became_active":"2024-04-25T00:55:38.633760+0000","last_became_peered":"2024-04-25T00:55:38.633760+0000","last_unstale":"2024-04-25T00:55:40.652622+0000","last_undegraded":"2024-04-25T00:55:40.652622+0000","last_fullsized":"2024-04-25T00:55:40.652622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637854+0000","last_change":"2024-04-25T00:55:39.644911+0000","last_active":"2024-04-25T00:55:40.637854+0000","last_peered":"2024-04-25T00:55:40.637854+0000","last_clean":"2024-04-25T00:55:40.637854+0000","last_became_active":"2024-04-25T00:55:39.644568+0000","last_became_peered":"2024-04-25T00:55:39.644568+0000","last_unstale":"2024-04-25T00:55:40.637854+0000","last_undegraded":"2024-04-25T00:55:40.637854+0000","last_fullsized":"2024-04-25T00:55:40.637854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:16.517320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637925+0000","last_change":"2024-04-25T00:55:38.637548+0000","last_active":"2024-04-25T00:55:40.637925+0000","last_peered":"2024-04-25T00:55:40.637925+0000","last_clean":"2024-04-25T00:55:40.637925+0000","last_became_active":"2024-04-25T00:55:38.636962+0000","last_became_peered":"2024-04-25T00:55:38.636962+0000","last_unstale":"2024-04-25T00:55:40.637925+0000","last_undegraded":"2024-04-25T00:55:40.637925+0000","last_fullsized":"2024-04-25T00:55:40.637925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020866+0000","last_change":"2024-04-25T00:55:39.628306+0000","last_active":"2024-04-25T00:55:41.020866+0000","last_peered":"2024-04-25T00:55:41.020866+0000","last_clean":"2024-04-25T00:55:41.020866+0000","last_became_active":"2024-04-25T00:55:39.627530+0000","last_became_peered":"2024-04-25T00:55:39.627530+0000","last_unstale":"2024-04-25T00:55:41.020866+0000","last_undegraded":"2024-04-25T00:55:41.020866+0000","last_fullsized":"2024-04-25T00:55:41.020866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637335+0000","last_change":"2024-04-25T00:55:38.640365+0000","last_active":"2024-04-25T00:55:40.637335+0000","last_peered":"2024-04-25T00:55:40.637335+0000","last_clean":"2024-04-25T00:55:40.637335+0000","last_became_active":"2024-04-25T00:55:38.639988+0000","last_became_peered":"2024-04-25T00:55:38.639988+0000","last_unstale":"2024-04-25T00:55:40.637335+0000","last_undegraded":"2024-04-25T00:55:40.637335+0000","last_fullsized":"2024-04-25T00:55:40.637335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636444+0000","last_change":"2024-04-25T00:55:39.647034+0000","last_active":"2024-04-25T00:55:40.636444+0000","last_peered":"2024-04-25T00:55:40.636444+0000","last_clean":"2024-04-25T00:55:40.636444+0000","last_became_active":"2024-04-25T00:55:39.644279+0000","last_became_peered":"2024-04-25T00:55:39.644279+0000","last_unstale":"2024-04-25T00:55:40.636444+0000","last_undegraded":"2024-04-25T00:55:40.636444+0000","last_fullsized":"2024-04-25T00:55:40.636444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:21.525835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652840+0000","last_change":"2024-04-25T00:55:38.626291+0000","last_active":"2024-04-25T00:55:40.652840+0000","last_peered":"2024-04-25T00:55:40.652840+0000","last_clean":"2024-04-25T00:55:40.652840+0000","last_became_active":"2024-04-25T00:55:38.625530+0000","last_became_peered":"2024-04-25T00:55:38.625530+0000","last_unstale":"2024-04-25T00:55:40.652840+0000","last_undegraded":"2024-04-25T00:55:40.652840+0000","last_fullsized":"2024-04-25T00:55:40.652840+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017484+0000","last_change":"2024-04-25T00:55:39.649518+0000","last_active":"2024-04-25T00:55:41.017484+0000","last_peered":"2024-04-25T00:55:41.017484+0000","last_clean":"2024-04-25T00:55:41.017484+0000","last_became_active":"2024-04-25T00:55:39.648701+0000","last_became_peered":"2024-04-25T00:55:39.648701+0000","last_unstale":"2024-04-25T00:55:41.017484+0000","last_undegraded":"2024-04-25T00:55:41.017484+0000","last_fullsized":"2024-04-25T00:55:41.017484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.552805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020445+0000","last_change":"2024-04-25T00:55:38.610242+0000","last_active":"2024-04-25T00:55:41.020445+0000","last_peered":"2024-04-25T00:55:41.020445+0000","last_clean":"2024-04-25T00:55:41.020445+0000","last_became_active":"2024-04-25T00:55:38.609855+0000","last_became_peered":"2024-04-25T00:55:38.609855+0000","last_unstale":"2024-04-25T00:55:41.020445+0000","last_undegraded":"2024-04-25T00:55:41.020445+0000","last_fullsized":"2024-04-25T00:55:41.020445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019689+0000","last_change":"2024-04-25T00:55:39.646421+0000","last_active":"2024-04-25T00:55:41.019689+0000","last_peered":"2024-04-25T00:55:41.019689+0000","last_clean":"2024-04-25T00:55:41.019689+0000","last_became_active":"2024-04-25T00:55:39.646119+0000","last_became_peered":"2024-04-25T00:55:39.646119+0000","last_unstale":"2024-04-25T00:55:41.019689+0000","last_undegraded":"2024-04-25T00:55:41.019689+0000","last_fullsized":"2024-04-25T00:55:41.019689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638593+0000","last_change":"2024-04-25T00:55:38.632875+0000","last_active":"2024-04-25T00:55:40.638593+0000","last_peered":"2024-04-25T00:55:40.638593+0000","last_clean":"2024-04-25T00:55:40.638593+0000","last_became_active":"2024-04-25T00:55:38.631370+0000","last_became_peered":"2024-04-25T00:55:38.631370+0000","last_unstale":"2024-04-25T00:55:40.638593+0000","last_undegraded":"2024-04-25T00:55:40.638593+0000","last_fullsized":"2024-04-25T00:55:40.638593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017573+0000","last_change":"2024-04-25T00:55:39.661000+0000","last_active":"2024-04-25T00:55:41.017573+0000","last_peered":"2024-04-25T00:55:41.017573+0000","last_clean":"2024-04-25T00:55:41.017573+0000","last_became_active":"2024-04-25T00:55:39.660704+0000","last_became_peered":"2024-04-25T00:55:39.660704+0000","last_unstale":"2024-04-25T00:55:41.017573+0000","last_undegraded":"2024-04-25T00:55:41.017573+0000","last_fullsized":"2024-04-25T00:55:41.017573+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:03:16.517320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652441+0000","last_change":"2024-04-25T00:55:38.614039+0000","last_active":"2024-04-25T00:55:40.652441+0000","last_peered":"2024-04-25T00:55:40.652441+0000","last_clean":"2024-04-25T00:55:40.652441+0000","last_became_active":"2024-04-25T00:55:38.612513+0000","last_became_peered":"2024-04-25T00:55:38.612513+0000","last_unstale":"2024-04-25T00:55:40.652441+0000","last_undegraded":"2024-04-25T00:55:40.652441+0000","last_fullsized":"2024-04-25T00:55:40.652441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652439+0000","last_change":"2024-04-25T00:55:39.639974+0000","last_active":"2024-04-25T00:55:40.652439+0000","last_peered":"2024-04-25T00:55:40.652439+0000","last_clean":"2024-04-25T00:55:40.652439+0000","last_became_active":"2024-04-25T00:55:39.639397+0000","last_became_peered":"2024-04-25T00:55:39.639397+0000","last_unstale":"2024-04-25T00:55:40.652439+0000","last_undegraded":"2024-04-25T00:55:40.652439+0000","last_fullsized":"2024-04-25T00:55:40.652439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017966+0000","last_change":"2024-04-25T00:55:38.601709+0000","last_active":"2024-04-25T00:55:41.017966+0000","last_peered":"2024-04-25T00:55:41.017966+0000","last_clean":"2024-04-25T00:55:41.017966+0000","last_became_active":"2024-04-25T00:55:38.601079+0000","last_became_peered":"2024-04-25T00:55:38.601079+0000","last_unstale":"2024-04-25T00:55:41.017966+0000","last_undegraded":"2024-04-25T00:55:41.017966+0000","last_fullsized":"2024-04-25T00:55:41.017966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:59:50.369672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016947+0000","last_change":"2024-04-25T00:55:39.652284+0000","last_active":"2024-04-25T00:55:41.016947+0000","last_peered":"2024-04-25T00:55:41.016947+0000","last_clean":"2024-04-25T00:55:41.016947+0000","last_became_active":"2024-04-25T00:55:39.651442+0000","last_became_peered":"2024-04-25T00:55:39.651442+0000","last_unstale":"2024-04-25T00:55:41.016947+0000","last_undegraded":"2024-04-25T00:55:41.016947+0000","last_fullsized":"2024-04-25T00:55:41.016947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636948+0000","last_change":"2024-04-25T00:55:38.603875+0000","last_active":"2024-04-25T00:55:40.636948+0000","last_peered":"2024-04-25T00:55:40.636948+0000","last_clean":"2024-04-25T00:55:40.636948+0000","last_became_active":"2024-04-25T00:55:38.602412+0000","last_became_peered":"2024-04-25T00:55:38.602412+0000","last_unstale":"2024-04-25T00:55:40.636948+0000","last_undegraded":"2024-04-25T00:55:40.636948+0000","last_fullsized":"2024-04-25T00:55:40.636948+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:34.488773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018026+0000","last_change":"2024-04-25T00:55:39.651592+0000","last_active":"2024-04-25T00:55:41.018026+0000","last_peered":"2024-04-25T00:55:41.018026+0000","last_clean":"2024-04-25T00:55:41.018026+0000","last_became_active":"2024-04-25T00:55:39.651045+0000","last_became_peered":"2024-04-25T00:55:39.651045+0000","last_unstale":"2024-04-25T00:55:41.018026+0000","last_undegraded":"2024-04-25T00:55:41.018026+0000","last_fullsized":"2024-04-25T00:55:41.018026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:51.008333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018836+0000","last_change":"2024-04-25T00:55:38.613063+0000","last_active":"2024-04-25T00:55:41.018836+0000","last_peered":"2024-04-25T00:55:41.018836+0000","last_clean":"2024-04-25T00:55:41.018836+0000","last_became_active":"2024-04-25T00:55:38.612651+0000","last_became_peered":"2024-04-25T00:55:38.612651+0000","last_unstale":"2024-04-25T00:55:41.018836+0000","last_undegraded":"2024-04-25T00:55:41.018836+0000","last_fullsized":"2024-04-25T00:55:41.018836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:25.842602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018376+0000","last_change":"2024-04-25T00:55:39.661043+0000","last_active":"2024-04-25T00:55:41.018376+0000","last_peered":"2024-04-25T00:55:41.018376+0000","last_clean":"2024-04-25T00:55:41.018376+0000","last_became_active":"2024-04-25T00:55:39.660577+0000","last_became_peered":"2024-04-25T00:55:39.660577+0000","last_unstale":"2024-04-25T00:55:41.018376+0000","last_undegraded":"2024-04-25T00:55:41.018376+0000","last_fullsized":"2024-04-25T00:55:41.018376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:35.187193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637626+0000","last_change":"2024-04-25T00:55:38.610833+0000","last_active":"2024-04-25T00:55:40.637626+0000","last_peered":"2024-04-25T00:55:40.637626+0000","last_clean":"2024-04-25T00:55:40.637626+0000","last_became_active":"2024-04-25T00:55:38.610476+0000","last_became_peered":"2024-04-25T00:55:38.610476+0000","last_unstale":"2024-04-25T00:55:40.637626+0000","last_undegraded":"2024-04-25T00:55:40.637626+0000","last_fullsized":"2024-04-25T00:55:40.637626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638296+0000","last_change":"2024-04-25T00:55:39.643691+0000","last_active":"2024-04-25T00:55:40.638296+0000","last_peered":"2024-04-25T00:55:40.638296+0000","last_clean":"2024-04-25T00:55:40.638296+0000","last_became_active":"2024-04-25T00:55:39.643024+0000","last_became_peered":"2024-04-25T00:55:39.643024+0000","last_unstale":"2024-04-25T00:55:40.638296+0000","last_undegraded":"2024-04-25T00:55:40.638296+0000","last_fullsized":"2024-04-25T00:55:40.638296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:22.249474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.020063+0000","last_change":"2024-04-25T00:55:38.623842+0000","last_active":"2024-04-25T00:55:41.020063+0000","last_peered":"2024-04-25T00:55:41.020063+0000","last_clean":"2024-04-25T00:55:41.020063+0000","last_became_active":"2024-04-25T00:55:38.623514+0000","last_became_peered":"2024-04-25T00:55:38.623514+0000","last_unstale":"2024-04-25T00:55:41.020063+0000","last_undegraded":"2024-04-25T00:55:41.020063+0000","last_fullsized":"2024-04-25T00:55:41.020063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:37:45.874180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017354+0000","last_change":"2024-04-25T00:55:39.658238+0000","last_active":"2024-04-25T00:55:41.017354+0000","last_peered":"2024-04-25T00:55:41.017354+0000","last_clean":"2024-04-25T00:55:41.017354+0000","last_became_active":"2024-04-25T00:55:39.657889+0000","last_became_peered":"2024-04-25T00:55:39.657889+0000","last_unstale":"2024-04-25T00:55:41.017354+0000","last_undegraded":"2024-04-25T00:55:41.017354+0000","last_fullsized":"2024-04-25T00:55:41.017354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016737+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:55:41.016737+0000","last_peered":"2024-04-25T00:55:41.016737+0000","last_clean":"2024-04-25T00:55:41.016737+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:55:41.016737+0000","last_undegraded":"2024-04-25T00:55:41.016737+0000","last_fullsized":"2024-04-25T00:55:41.016737+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637451+0000","last_change":"2024-04-25T00:55:38.632700+0000","last_active":"2024-04-25T00:55:40.637451+0000","last_peered":"2024-04-25T00:55:40.637451+0000","last_clean":"2024-04-25T00:55:40.637451+0000","last_became_active":"2024-04-25T00:55:38.631901+0000","last_became_peered":"2024-04-25T00:55:38.631901+0000","last_unstale":"2024-04-25T00:55:40.637451+0000","last_undegraded":"2024-04-25T00:55:40.637451+0000","last_fullsized":"2024-04-25T00:55:40.637451+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021271+0000","last_change":"2024-04-25T00:55:39.639013+0000","last_active":"2024-04-25T00:55:41.021271+0000","last_peered":"2024-04-25T00:55:41.021271+0000","last_clean":"2024-04-25T00:55:41.021271+0000","last_became_active":"2024-04-25T00:55:39.638189+0000","last_became_peered":"2024-04-25T00:55:39.638189+0000","last_unstale":"2024-04-25T00:55:41.021271+0000","last_undegraded":"2024-04-25T00:55:41.021271+0000","last_fullsized":"2024-04-25T00:55:41.021271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:55:55.033708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016724+0000","last_change":"2024-04-25T00:55:38.621598+0000","last_active":"2024-04-25T00:55:41.016724+0000","last_peered":"2024-04-25T00:55:41.016724+0000","last_clean":"2024-04-25T00:55:41.016724+0000","last_became_active":"2024-04-25T00:55:38.620844+0000","last_became_peered":"2024-04-25T00:55:38.620844+0000","last_unstale":"2024-04-25T00:55:41.016724+0000","last_undegraded":"2024-04-25T00:55:41.016724+0000","last_fullsized":"2024-04-25T00:55:41.016724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.019793+0000","last_change":"2024-04-25T00:55:39.626650+0000","last_active":"2024-04-25T00:55:41.019793+0000","last_peered":"2024-04-25T00:55:41.019793+0000","last_clean":"2024-04-25T00:55:41.019793+0000","last_became_active":"2024-04-25T00:55:39.625876+0000","last_became_peered":"2024-04-25T00:55:39.625876+0000","last_unstale":"2024-04-25T00:55:41.019793+0000","last_undegraded":"2024-04-25T00:55:41.019793+0000","last_fullsized":"2024-04-25T00:55:41.019793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018447+0000","last_change":"2024-04-25T00:55:38.605319+0000","last_active":"2024-04-25T00:55:41.018447+0000","last_peered":"2024-04-25T00:55:41.018447+0000","last_clean":"2024-04-25T00:55:41.018447+0000","last_became_active":"2024-04-25T00:55:38.604920+0000","last_became_peered":"2024-04-25T00:55:38.604920+0000","last_unstale":"2024-04-25T00:55:41.018447+0000","last_undegraded":"2024-04-25T00:55:41.018447+0000","last_fullsized":"2024-04-25T00:55:41.018447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636431+0000","last_change":"2024-04-25T00:55:39.635799+0000","last_active":"2024-04-25T00:55:40.636431+0000","last_peered":"2024-04-25T00:55:40.636431+0000","last_clean":"2024-04-25T00:55:40.636431+0000","last_became_active":"2024-04-25T00:55:39.633616+0000","last_became_peered":"2024-04-25T00:55:39.633616+0000","last_unstale":"2024-04-25T00:55:40.636431+0000","last_undegraded":"2024-04-25T00:55:40.636431+0000","last_fullsized":"2024-04-25T00:55:40.636431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:04.370694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638712+0000","last_change":"2024-04-25T00:55:38.638178+0000","last_active":"2024-04-25T00:55:40.638712+0000","last_peered":"2024-04-25T00:55:40.638712+0000","last_clean":"2024-04-25T00:55:40.638712+0000","last_became_active":"2024-04-25T00:55:38.637419+0000","last_became_peered":"2024-04-25T00:55:38.637419+0000","last_unstale":"2024-04-25T00:55:40.638712+0000","last_undegraded":"2024-04-25T00:55:40.638712+0000","last_fullsized":"2024-04-25T00:55:40.638712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636532+0000","last_change":"2024-04-25T00:55:39.635022+0000","last_active":"2024-04-25T00:55:40.636532+0000","last_peered":"2024-04-25T00:55:40.636532+0000","last_clean":"2024-04-25T00:55:40.636532+0000","last_became_active":"2024-04-25T00:55:39.634107+0000","last_became_peered":"2024-04-25T00:55:39.634107+0000","last_unstale":"2024-04-25T00:55:40.636532+0000","last_undegraded":"2024-04-25T00:55:40.636532+0000","last_fullsized":"2024-04-25T00:55:40.636532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.087648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637039+0000","last_change":"2024-04-25T00:55:38.607260+0000","last_active":"2024-04-25T00:55:40.637039+0000","last_peered":"2024-04-25T00:55:40.637039+0000","last_clean":"2024-04-25T00:55:40.637039+0000","last_became_active":"2024-04-25T00:55:38.606910+0000","last_became_peered":"2024-04-25T00:55:38.606910+0000","last_unstale":"2024-04-25T00:55:40.637039+0000","last_undegraded":"2024-04-25T00:55:40.637039+0000","last_fullsized":"2024-04-25T00:55:40.637039+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:25.842602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637823+0000","last_change":"2024-04-25T00:55:39.641228+0000","last_active":"2024-04-25T00:55:40.637823+0000","last_peered":"2024-04-25T00:55:40.637823+0000","last_clean":"2024-04-25T00:55:40.637823+0000","last_became_active":"2024-04-25T00:55:39.640434+0000","last_became_peered":"2024-04-25T00:55:39.640434+0000","last_unstale":"2024-04-25T00:55:40.637823+0000","last_undegraded":"2024-04-25T00:55:40.637823+0000","last_fullsized":"2024-04-25T00:55:40.637823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:08:32.552805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018046+0000","last_change":"2024-04-25T00:55:38.630316+0000","last_active":"2024-04-25T00:55:41.018046+0000","last_peered":"2024-04-25T00:55:41.018046+0000","last_clean":"2024-04-25T00:55:41.018046+0000","last_became_active":"2024-04-25T00:55:38.629417+0000","last_became_peered":"2024-04-25T00:55:38.629417+0000","last_unstale":"2024-04-25T00:55:41.018046+0000","last_undegraded":"2024-04-25T00:55:41.018046+0000","last_fullsized":"2024-04-25T00:55:41.018046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636661+0000","last_change":"2024-04-25T00:55:39.633030+0000","last_active":"2024-04-25T00:55:40.636661+0000","last_peered":"2024-04-25T00:55:40.636661+0000","last_clean":"2024-04-25T00:55:40.636661+0000","last_became_active":"2024-04-25T00:55:39.632300+0000","last_became_peered":"2024-04-25T00:55:39.632300+0000","last_unstale":"2024-04-25T00:55:40.636661+0000","last_undegraded":"2024-04-25T00:55:40.636661+0000","last_fullsized":"2024-04-25T00:55:40.636661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:50:34.306408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652457+0000","last_change":"2024-04-25T00:55:38.634050+0000","last_active":"2024-04-25T00:55:40.652457+0000","last_peered":"2024-04-25T00:55:40.652457+0000","last_clean":"2024-04-25T00:55:40.652457+0000","last_became_active":"2024-04-25T00:55:38.633672+0000","last_became_peered":"2024-04-25T00:55:38.633672+0000","last_unstale":"2024-04-25T00:55:40.652457+0000","last_undegraded":"2024-04-25T00:55:40.652457+0000","last_fullsized":"2024-04-25T00:55:40.652457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.022010+0000","last_change":"2024-04-25T00:55:39.652710+0000","last_active":"2024-04-25T00:55:41.022010+0000","last_peered":"2024-04-25T00:55:41.022010+0000","last_clean":"2024-04-25T00:55:41.022010+0000","last_became_active":"2024-04-25T00:55:39.652302+0000","last_became_peered":"2024-04-25T00:55:39.652302+0000","last_unstale":"2024-04-25T00:55:41.022010+0000","last_undegraded":"2024-04-25T00:55:41.022010+0000","last_fullsized":"2024-04-25T00:55:41.022010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652764+0000","last_change":"2024-04-25T00:55:38.636274+0000","last_active":"2024-04-25T00:55:40.652764+0000","last_peered":"2024-04-25T00:55:40.652764+0000","last_clean":"2024-04-25T00:55:40.652764+0000","last_became_active":"2024-04-25T00:55:38.635626+0000","last_became_peered":"2024-04-25T00:55:38.635626+0000","last_unstale":"2024-04-25T00:55:40.652764+0000","last_undegraded":"2024-04-25T00:55:40.652764+0000","last_fullsized":"2024-04-25T00:55:40.652764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018903+0000","last_change":"2024-04-25T00:55:39.635041+0000","last_active":"2024-04-25T00:55:41.018903+0000","last_peered":"2024-04-25T00:55:41.018903+0000","last_clean":"2024-04-25T00:55:41.018903+0000","last_became_active":"2024-04-25T00:55:39.634758+0000","last_became_peered":"2024-04-25T00:55:39.634758+0000","last_unstale":"2024-04-25T00:55:41.018903+0000","last_undegraded":"2024-04-25T00:55:41.018903+0000","last_fullsized":"2024-04-25T00:55:41.018903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:53.762656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.016600+0000","last_change":"2024-04-25T00:55:38.630288+0000","last_active":"2024-04-25T00:55:41.016600+0000","last_peered":"2024-04-25T00:55:41.016600+0000","last_clean":"2024-04-25T00:55:41.016600+0000","last_became_active":"2024-04-25T00:55:38.629355+0000","last_became_peered":"2024-04-25T00:55:38.629355+0000","last_unstale":"2024-04-25T00:55:41.016600+0000","last_undegraded":"2024-04-25T00:55:41.016600+0000","last_fullsized":"2024-04-25T00:55:41.016600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:48.161363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636226+0000","last_change":"2024-04-25T00:55:39.623782+0000","last_active":"2024-04-25T00:55:40.636226+0000","last_peered":"2024-04-25T00:55:40.636226+0000","last_clean":"2024-04-25T00:55:40.636226+0000","last_became_active":"2024-04-25T00:55:39.622815+0000","last_became_peered":"2024-04-25T00:55:39.622815+0000","last_unstale":"2024-04-25T00:55:40.636226+0000","last_undegraded":"2024-04-25T00:55:40.636226+0000","last_fullsized":"2024-04-25T00:55:40.636226+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:03.896351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637109+0000","last_change":"2024-04-25T00:55:38.618497+0000","last_active":"2024-04-25T00:55:40.637109+0000","last_peered":"2024-04-25T00:55:40.637109+0000","last_clean":"2024-04-25T00:55:40.637109+0000","last_became_active":"2024-04-25T00:55:38.618150+0000","last_became_peered":"2024-04-25T00:55:38.618150+0000","last_unstale":"2024-04-25T00:55:40.637109+0000","last_undegraded":"2024-04-25T00:55:40.637109+0000","last_fullsized":"2024-04-25T00:55:40.637109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:02.885821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021086+0000","last_change":"2024-04-25T00:55:39.640734+0000","last_active":"2024-04-25T00:55:41.021086+0000","last_peered":"2024-04-25T00:55:41.021086+0000","last_clean":"2024-04-25T00:55:41.021086+0000","last_became_active":"2024-04-25T00:55:39.639372+0000","last_became_peered":"2024-04-25T00:55:39.639372+0000","last_unstale":"2024-04-25T00:55:41.021086+0000","last_undegraded":"2024-04-25T00:55:41.021086+0000","last_fullsized":"2024-04-25T00:55:41.021086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:24:23.334376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.636950+0000","last_change":"2024-04-25T00:55:38.613079+0000","last_active":"2024-04-25T00:55:40.636950+0000","last_peered":"2024-04-25T00:55:40.636950+0000","last_clean":"2024-04-25T00:55:40.636950+0000","last_became_active":"2024-04-25T00:55:38.612643+0000","last_became_peered":"2024-04-25T00:55:38.612643+0000","last_unstale":"2024-04-25T00:55:40.636950+0000","last_undegraded":"2024-04-25T00:55:40.636950+0000","last_fullsized":"2024-04-25T00:55:40.636950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:20.515305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021795+0000","last_change":"2024-04-25T00:55:39.640875+0000","last_active":"2024-04-25T00:55:41.021795+0000","last_peered":"2024-04-25T00:55:41.021795+0000","last_clean":"2024-04-25T00:55:41.021795+0000","last_became_active":"2024-04-25T00:55:39.638658+0000","last_became_peered":"2024-04-25T00:55:39.638658+0000","last_unstale":"2024-04-25T00:55:41.021795+0000","last_undegraded":"2024-04-25T00:55:41.021795+0000","last_fullsized":"2024-04-25T00:55:41.021795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017957+0000","last_change":"2024-04-25T00:55:38.624428+0000","last_active":"2024-04-25T00:55:41.017957+0000","last_peered":"2024-04-25T00:55:41.017957+0000","last_clean":"2024-04-25T00:55:41.017957+0000","last_became_active":"2024-04-25T00:55:38.622736+0000","last_became_peered":"2024-04-25T00:55:38.622736+0000","last_unstale":"2024-04-25T00:55:41.017957+0000","last_undegraded":"2024-04-25T00:55:41.017957+0000","last_fullsized":"2024-04-25T00:55:41.017957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:26.077118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.652902+0000","last_change":"2024-04-25T00:55:39.631349+0000","last_active":"2024-04-25T00:55:40.652902+0000","last_peered":"2024-04-25T00:55:40.652902+0000","last_clean":"2024-04-25T00:55:40.652902+0000","last_became_active":"2024-04-25T00:55:39.630507+0000","last_became_peered":"2024-04-25T00:55:39.630507+0000","last_unstale":"2024-04-25T00:55:40.652902+0000","last_undegraded":"2024-04-25T00:55:40.652902+0000","last_fullsized":"2024-04-25T00:55:40.652902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:13.518623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.637806+0000","last_change":"2024-04-25T00:55:39.628089+0000","last_active":"2024-04-25T00:55:40.637806+0000","last_peered":"2024-04-25T00:55:40.637806+0000","last_clean":"2024-04-25T00:55:40.637806+0000","last_became_active":"2024-04-25T00:55:39.627070+0000","last_became_peered":"2024-04-25T00:55:39.627070+0000","last_unstale":"2024-04-25T00:55:40.637806+0000","last_undegraded":"2024-04-25T00:55:40.637806+0000","last_fullsized":"2024-04-25T00:55:40.637806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:27.888531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017214+0000","last_change":"2024-04-25T00:55:38.634723+0000","last_active":"2024-04-25T00:55:41.017214+0000","last_peered":"2024-04-25T00:55:41.017214+0000","last_clean":"2024-04-25T00:55:41.017214+0000","last_became_active":"2024-04-25T00:55:38.633867+0000","last_became_peered":"2024-04-25T00:55:38.633867+0000","last_unstale":"2024-04-25T00:55:41.017214+0000","last_undegraded":"2024-04-25T00:55:41.017214+0000","last_fullsized":"2024-04-25T00:55:41.017214+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:03:08.252396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:40.638050+0000","last_change":"2024-04-25T00:55:39.618027+0000","last_active":"2024-04-25T00:55:40.638050+0000","last_peered":"2024-04-25T00:55:40.638050+0000","last_clean":"2024-04-25T00:55:40.638050+0000","last_became_active":"2024-04-25T00:55:39.617211+0000","last_became_peered":"2024-04-25T00:55:39.617211+0000","last_unstale":"2024-04-25T00:55:40.638050+0000","last_undegraded":"2024-04-25T00:55:40.638050+0000","last_fullsized":"2024-04-25T00:55:40.638050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:11.034367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.017702+0000","last_change":"2024-04-25T00:55:38.636516+0000","last_active":"2024-04-25T00:55:41.017702+0000","last_peered":"2024-04-25T00:55:41.017702+0000","last_clean":"2024-04-25T00:55:41.017702+0000","last_became_active":"2024-04-25T00:55:38.635662+0000","last_became_peered":"2024-04-25T00:55:38.635662+0000","last_unstale":"2024-04-25T00:55:41.017702+0000","last_undegraded":"2024-04-25T00:55:41.017702+0000","last_fullsized":"2024-04-25T00:55:41.017702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:48:55.776052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.018504+0000","last_change":"2024-04-25T00:55:39.622594+0000","last_active":"2024-04-25T00:55:41.018504+0000","last_peered":"2024-04-25T00:55:41.018504+0000","last_clean":"2024-04-25T00:55:41.018504+0000","last_became_active":"2024-04-25T00:55:39.621768+0000","last_became_peered":"2024-04-25T00:55:39.621768+0000","last_unstale":"2024-04-25T00:55:41.018504+0000","last_undegraded":"2024-04-25T00:55:41.018504+0000","last_fullsized":"2024-04-25T00:55:41.018504+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:27.074899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021808+0000","last_change":"2024-04-25T00:55:38.609758+0000","last_active":"2024-04-25T00:55:41.021808+0000","last_peered":"2024-04-25T00:55:41.021808+0000","last_clean":"2024-04-25T00:55:41.021808+0000","last_became_active":"2024-04-25T00:55:38.609401+0000","last_became_peered":"2024-04-25T00:55:38.609401+0000","last_unstale":"2024-04-25T00:55:41.021808+0000","last_undegraded":"2024-04-25T00:55:41.021808+0000","last_fullsized":"2024-04-25T00:55:41.021808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:01.720059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021639+0000","last_change":"2024-04-25T00:55:39.657195+0000","last_active":"2024-04-25T00:55:41.021639+0000","last_peered":"2024-04-25T00:55:41.021639+0000","last_clean":"2024-04-25T00:55:41.021639+0000","last_became_active":"2024-04-25T00:55:39.656921+0000","last_became_peered":"2024-04-25T00:55:39.656921+0000","last_unstale":"2024-04-25T00:55:41.021639+0000","last_undegraded":"2024-04-25T00:55:41.021639+0000","last_fullsized":"2024-04-25T00:55:41.021639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:38.577763+0000","last_clean_scrub_stamp":"2024-04-25T00:55:38.577763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:31:37.412140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T00:55:41.021304+0000","last_change":"2024-04-25T00:55:38.629445+0000","last_active":"2024-04-25T00:55:41.021304+0000","last_peered":"2024-04-25T00:55:41.021304+0000","last_clean":"2024-04-25T00:55:41.021304+0000","last_became_active":"2024-04-25T00:55:38.628997+0000","last_became_peered":"2024-04-25T00:55:38.628997+0000","last_unstale":"2024-04-25T00:55:41.021304+0000","last_undegraded":"2024-04-25T00:55:41.021304+0000","last_fullsized":"2024-04-25T00:55:41.021304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:37.567233+0000","last_clean_scrub_stamp":"2024-04-25T00:55:37.567233+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:22:01.720059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:55:46.573 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:55:46.573 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:55:46.573 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:55:46.573 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T00:55:46.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba33392640 1 Processor -- start 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba33392640 1 -- start start 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba33392640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0ae3b0 0x7fba2c0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba33392640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba2c0b2780 0x7fba2c14e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba33392640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba2c0b52f0 con 0x7fba2c0adfd0 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba33392640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fba2c0af0b0 con 0x7fba2c0b2780 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba33392640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba2c0b0800 con 0x7fba2c0ae3b0 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba3192f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba2c0b2780 0x7fba2c14e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba3192f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fba2c0b2780 0x7fba2c14e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51386/0 (socket says 172.21.15.112:51386) 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba3192f640 1 -- 172.21.15.112:0/4053258102 learned_addr learned my addr 172.21.15.112:0/4053258102 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba3092d640 1 --2- 172.21.15.112:0/4053258102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0ae3b0 0x7fba2c0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba3192f640 1 -- 172.21.15.112:0/4053258102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0ae3b0 msgr2=0x7fba2c0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba3192f640 1 --2- 172.21.15.112:0/4053258102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0ae3b0 0x7fba2c0b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba3192f640 1 -- 172.21.15.112:0/4053258102 >> v1:172.21.15.187:6789/0 conn(0x7fba2c0adfd0 legacy=0x7fba2c0ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:46.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.716+0000 7fba3192f640 1 -- 172.21.15.112:0/4053258102 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c143f50 con 0x7fba2c0b2780 2024-04-25T00:55:46.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba3192f640 1 --2- 172.21.15.112:0/4053258102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0b2780 0x7fba2c14e1d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba24009e80 tx=0x7fba2401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81899e84de24754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba23fff640 1 -- 172.21.15.112:0/4053258102 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240a5540 con 0x7fba2c0b2780 2024-04-25T00:55:46.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba23fff640 1 -- 172.21.15.112:0/4053258102 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba240a56e0 con 0x7fba2c0b2780 2024-04-25T00:55:46.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba23fff640 1 -- 172.21.15.112:0/4053258102 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba240a5ab0 con 0x7fba2c0b2780 2024-04-25T00:55:46.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 -- 172.21.15.112:0/4053258102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0b2780 msgr2=0x7fba2c14e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 --2- 172.21.15.112:0/4053258102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0b2780 0x7fba2c14e1d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fba24009e80 tx=0x7fba2401c940 comp rx=0 tx=0).stop 2024-04-25T00:55:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 -- 172.21.15.112:0/4053258102 shutdown_connections 2024-04-25T00:55:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 --2- 172.21.15.112:0/4053258102 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0b2780 0x7fba2c14e1d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 --2- 172.21.15.112:0/4053258102 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0ae3b0 0x7fba2c0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 -- 172.21.15.112:0/4053258102 >> 172.21.15.112:0/4053258102 conn(0x7fba2c0b5980 msgr2=0x7fba2c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 -- 172.21.15.112:0/4053258102 shutdown_connections 2024-04-25T00:55:46.725 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 -- 172.21.15.112:0/4053258102 wait complete. 2024-04-25T00:55:46.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 Processor -- start 2024-04-25T00:55:46.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.720+0000 7fba33392640 1 -- start start 2024-04-25T00:55:46.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0adfd0 0x7fba2c1412b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2c0ae3b0 0x7fba2c071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0b2780 0x7fba2c0722e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c152bc0 con 0x7fba2c0adfd0 2024-04-25T00:55:46.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c152a40 con 0x7fba2c0ae3b0 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fba2c152d40 con 0x7fba2c0b2780 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0adfd0 0x7fba2c1412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0adfd0 0x7fba2c1412b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51396/0 (socket says 172.21.15.112:51396) 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 -- 172.21.15.112:0/1792124265 learned_addr learned my addr 172.21.15.112:0/1792124265 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3092d640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2c0ae3b0 0x7fba2c071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3192f640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0b2780 0x7fba2c0722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 -- 172.21.15.112:0/1792124265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0b2780 msgr2=0x7fba2c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0b2780 0x7fba2c0722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 -- 172.21.15.112:0/1792124265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2c0ae3b0 msgr2=0x7fba2c071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2c0ae3b0 0x7fba2c071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 -- 172.21.15.112:0/1792124265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c1436f0 con 0x7fba2c0adfd0 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3112e640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0adfd0 0x7fba2c1412b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c450 tx=0x7fba1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba21ffb640 1 -- 172.21.15.112:0/1792124265 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012830 con 0x7fba2c0adfd0 2024-04-25T00:55:46.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba21ffb640 1 -- 172.21.15.112:0/1792124265 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1c017070 con 0x7fba2c0adfd0 2024-04-25T00:55:46.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba21ffb640 1 -- 172.21.15.112:0/1792124265 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012c30 con 0x7fba2c0adfd0 2024-04-25T00:55:46.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2c06f650 con 0x7fba2c0adfd0 2024-04-25T00:55:46.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2c141d30 con 0x7fba2c0adfd0 2024-04-25T00:55:46.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2c04f580 con 0x7fba2c0adfd0 2024-04-25T00:55:46.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba21ffb640 1 -- 172.21.15.112:0/1792124265 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fba1c01c460 con 0x7fba2c0adfd0 2024-04-25T00:55:46.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba21ffb640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fba100678f0 0x7fba10069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:46.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba21ffb640 1 -- 172.21.15.112:0/1792124265 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fba1c085930 con 0x7fba2c0adfd0 2024-04-25T00:55:46.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3092d640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fba100678f0 0x7fba10069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:46.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.724+0000 7fba3092d640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fba100678f0 0x7fba10069db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0aff80 tx=0x7fba28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:46.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.728+0000 7fba21ffb640 1 -- 172.21.15.112:0/1792124265 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba1c056500 con 0x7fba2c0adfd0 2024-04-25T00:55:46.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.912+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fba2c06f370 con 0x7fba2c0adfd0 2024-04-25T00:55:46.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.912+0000 7fba21ffb640 1 -- 172.21.15.112:0/1792124265 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fba1c055e00 con 0x7fba2c0adfd0 2024-04-25T00:55:46.916 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:46.916 INFO:teuthology.orchestra.run.smithi112.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:55:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fba100678f0 msgr2=0x7fba10069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fba100678f0 0x7fba10069db0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0aff80 tx=0x7fba28008040 comp rx=0 tx=0).stop 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0adfd0 msgr2=0x7fba2c1412b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0adfd0 0x7fba2c1412b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00c450 tx=0x7fba1c00c910 comp rx=0 tx=0).stop 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 shutdown_connections 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fba2c0b2780 0x7fba2c0722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fba100678f0 0x7fba10069db0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fba2c0ae3b0 0x7fba2c071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 --2- 172.21.15.112:0/1792124265 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba2c0adfd0 0x7fba2c1412b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 >> 172.21.15.112:0/1792124265 conn(0x7fba2c0b5980 msgr2=0x7fba2c0b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 shutdown_connections 2024-04-25T00:55:46.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:46.916+0000 7fba33392640 1 -- 172.21.15.112:0/1792124265 wait complete. 2024-04-25T00:55:46.934 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:55:46.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:47.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.044+0000 7f025682b640 1 Processor -- start 2024-04-25T00:55:47.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.044+0000 7f025682b640 1 -- start start 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f025682b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02500adfd0 0x7f02500ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f025682b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f02500ae9a0 0x7f02500b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f025682b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02500b52f0 con 0x7f02500aee00 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f025682b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f02500b0800 con 0x7f02500ae9a0 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f025682b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0250143f50 con 0x7f02500adfd0 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f0254dc8640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f02500aee00 0x7f02500b2780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57314/0 (socket says 172.21.15.112:57314) 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f0254dc8640 1 -- 172.21.15.112:0/241068618 learned_addr learned my addr 172.21.15.112:0/241068618 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024ffff640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02500adfd0 0x7f02500ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2584998654 0 0) 0x7f02500b52f0 con 0x7f02500aee00 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024f7fe640 1 --2- 172.21.15.112:0/241068618 >> v2:172.21.15.187:3300/0 conn(0x7f02500ae9a0 0x7f02500b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0238003530 con 0x7f02500aee00 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 760454036 0 0) 0x7f0238003530 con 0x7f02500aee00 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02500adfd0 msgr2=0x7f02500ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02500adfd0 0x7f02500ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 >> v2:172.21.15.187:3300/0 conn(0x7f02500ae9a0 msgr2=0x7f02500b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 --2- 172.21.15.112:0/241068618 >> v2:172.21.15.187:3300/0 conn(0x7f02500ae9a0 0x7f02500b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0250144230 con 0x7f02500aee00 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f02480016a0 con 0x7f02500aee00 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 >> v1:172.21.15.112:6789/0 conn(0x7f02500aee00 legacy=0x7f02500b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380039c0 0x7f0238003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0238004810 0x7f0238004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02500adfd0 0x7f02380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f02500b0800 con 0x7f0238004810 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0250143f50 con 0x7f02380039c0 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f023800ba20 con 0x7f02500adfd0 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024f7fe640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0238004810 0x7f0238004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024ffff640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380039c0 0x7f0238003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f0254dc8640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02500adfd0 0x7f02380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024f7fe640 1 -- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02500adfd0 msgr2=0x7f02380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024f7fe640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f02500adfd0 0x7f02380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024f7fe640 1 -- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380039c0 msgr2=0x7f0238003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024f7fe640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02380039c0 0x7f0238003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024f7fe640 1 -- 172.21.15.112:0/241068618 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0238003530 con 0x7f0238004810 2024-04-25T00:55:47.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024f7fe640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0238004810 0x7f0238004c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0244009b10 tx=0x7f024401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3fb1ce0665fae24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02440096b0 con 0x7f0238004810 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02440a8420 con 0x7f0238004810 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.048+0000 7f024effd640 1 -- 172.21.15.112:0/241068618 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02440a8810 con 0x7f0238004810 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- 172.21.15.112:0/241068618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0238004810 msgr2=0x7f0238004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 --2- 172.21.15.112:0/241068618 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0238004810 0x7f0238004c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0244009b10 tx=0x7f024401cab0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 -- 172.21.15.112:0/241068618 reap_dead start 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- 172.21.15.112:0/241068618 shutdown_connections 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- 172.21.15.112:0/241068618 >> 172.21.15.112:0/241068618 conn(0x7f02500b5980 msgr2=0x7f02500b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- 172.21.15.112:0/241068618 shutdown_connections 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- 172.21.15.112:0/241068618 wait complete. 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 Processor -- start 2024-04-25T00:55:47.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- start start 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02501413f0 0x7f025014a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025014ceb0 0x7f02501e1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02501417d0 0x7f02501e1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f025014e350 con 0x7f02501417d0 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f025014e1d0 con 0x7f02501413f0 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f025014e4d0 con 0x7f025014ceb0 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02501417d0 0x7f02501e1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02501417d0 0x7f02501e1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51410/0 (socket says 172.21.15.112:51410) 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 -- 172.21.15.112:0/2713969956 learned_addr learned my addr 172.21.15.112:0/2713969956 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:47.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f024f7fe640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025014ceb0 0x7f02501e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f024ffff640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02501413f0 0x7f025014a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 -- 172.21.15.112:0/2713969956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025014ceb0 msgr2=0x7f02501e1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025014ceb0 0x7f02501e1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 -- 172.21.15.112:0/2713969956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02501413f0 msgr2=0x7f025014a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02501413f0 0x7f025014a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 -- 172.21.15.112:0/2713969956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025014c410 con 0x7f02501417d0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f0254dc8640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02501417d0 0x7f02501e1570 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f025006dc60 tx=0x7f024800e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f024cff9640 1 -- 172.21.15.112:0/2713969956 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248019070 con 0x7f02501417d0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f024cff9640 1 -- 172.21.15.112:0/2713969956 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0248010510 con 0x7f02501417d0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025014c6b0 con 0x7f02501417d0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f024cff9640 1 -- 172.21.15.112:0/2713969956 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480108e0 con 0x7f02501417d0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0250145de0 con 0x7f02501417d0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.052+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025004f580 con 0x7f02501417d0 2024-04-25T00:55:47.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.056+0000 7f024cff9640 1 -- 172.21.15.112:0/2713969956 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f024801c070 con 0x7f02501417d0 2024-04-25T00:55:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.056+0000 7f024cff9640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f023c0678c0 0x7f023c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.056+0000 7f024cff9640 1 -- 172.21.15.112:0/2713969956 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f02480850c0 con 0x7f02501417d0 2024-04-25T00:55:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.056+0000 7f024ffff640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f023c0678c0 0x7f023c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.056+0000 7f024ffff640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f023c0678c0 0x7f023c069d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0240007440 tx=0x7f024000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.060+0000 7f024cff9640 1 -- 172.21.15.112:0/2713969956 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0248055c90 con 0x7f02501417d0 2024-04-25T00:55:47.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.204+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f025006b9c0 con 0x7f02501417d0 2024-04-25T00:55:47.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f024cff9640 1 -- 172.21.15.112:0/2713969956 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f0248055590 con 0x7f02501417d0 2024-04-25T00:55:47.211 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:47.211 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:47.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f023c0678c0 msgr2=0x7f023c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f023c0678c0 0x7f023c069d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0240007440 tx=0x7f024000b040 comp rx=0 tx=0).stop 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02501417d0 msgr2=0x7f02501e1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02501417d0 0x7f02501e1570 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f025006dc60 tx=0x7f024800e8a0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 shutdown_connections 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02501417d0 0x7f02501e1570 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f025014ceb0 0x7f02501e1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f023c0678c0 0x7f023c069d80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 --2- 172.21.15.112:0/2713969956 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f02501413f0 0x7f025014a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 >> 172.21.15.112:0/2713969956 conn(0x7f02500b5980 msgr2=0x7f025014e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 shutdown_connections 2024-04-25T00:55:47.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.208+0000 7f025682b640 1 -- 172.21.15.112:0/2713969956 wait complete. 2024-04-25T00:55:47.229 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:47.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.352+0000 7f35f3f31640 1 Processor -- start 2024-04-25T00:55:47.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.352+0000 7f35f3f31640 1 -- start start 2024-04-25T00:55:47.361 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f3f31640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f35ec06caf0 0x7f35ec0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f3f31640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ec0b42d0 con 0x7f35ec09ded0 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f3f31640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ec09e2b0 con 0x7f35ec0add40 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f3f31640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f35ec0ae790 con 0x7f35ec06caf0 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f1ccd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f35ec09ded0 0x7f35ec0b34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57320/0 (socket says 172.21.15.112:57320) 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f1ccd640 1 -- 172.21.15.112:0/517188851 learned_addr learned my addr 172.21.15.112:0/517188851 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2142735447 0 0) 0x7f35ec0b42d0 con 0x7f35ec09ded0 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35d8003530 con 0x7f35ec09ded0 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 --2- 172.21.15.112:0/517188851 >> v2:172.21.15.187:3300/0 conn(0x7f35ec06caf0 0x7f35ec0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3582686330 0 0) 0x7f35d8003530 con 0x7f35ec09ded0 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 >> v2:172.21.15.187:3300/0 conn(0x7f35ec06caf0 msgr2=0x7f35ec0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 --2- 172.21.15.112:0/517188851 >> v2:172.21.15.187:3300/0 conn(0x7f35ec06caf0 0x7f35ec0b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.362 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 >> v1:172.21.15.187:6789/0 conn(0x7f35ec0add40 legacy=0x7f35ec0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec0aea70 con 0x7f35ec09ded0 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f35e8001680 con 0x7f35ec09ded0 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 >> v1:172.21.15.112:6789/0 conn(0x7f35ec09ded0 legacy=0x7f35ec0b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35d80039c0 0x7f35d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35d8004810 0x7f35d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35d800cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0ae790 con 0x7f35d80051d0 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f35d800d280 con 0x7f35d80039c0 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f35d800d480 con 0x7f35d8004810 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35d800cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f1ccd640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35d80039c0 0x7f35d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f14cc640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35d8004810 0x7f35d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 -- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35d8004810 msgr2=0x7f35d8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35d8004810 0x7f35d8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 -- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35d80039c0 msgr2=0x7f35d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35d80039c0 0x7f35d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 -- 172.21.15.112:0/517188851 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d8003530 con 0x7f35d80051d0 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f1ccd640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35d80039c0 0x7f35d8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 -- 172.21.15.112:0/517188851 reap_dead start 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f24ce640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35d800cae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009fa0 tx=0x7f35e401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110060b59b99280e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e40b2070 con 0x7f35d80051d0 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e40a5d50 con 0x7f35d80051d0 2024-04-25T00:55:47.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.356+0000 7f35f0ccb640 1 -- 172.21.15.112:0/517188851 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e4009bf0 con 0x7f35d80051d0 2024-04-25T00:55:47.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- 172.21.15.112:0/517188851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 msgr2=0x7f35d800cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35d800cae0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009fa0 tx=0x7f35e401cc80 comp rx=0 tx=0).stop 2024-04-25T00:55:47.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- 172.21.15.112:0/517188851 shutdown_connections 2024-04-25T00:55:47.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 --2- 172.21.15.112:0/517188851 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35d800cae0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- 172.21.15.112:0/517188851 >> 172.21.15.112:0/517188851 conn(0x7f35ec0b8c80 msgr2=0x7f35ec0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:47.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- 172.21.15.112:0/517188851 shutdown_connections 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- 172.21.15.112:0/517188851 wait complete. 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 Processor -- start 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- start start 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35ec1ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35ec06f4d0 0x7f35ec09c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35ec06f910 0x7f35ec09c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0ae370 con 0x7f35d80051d0 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec0ada10 con 0x7f35ec06f4d0 2024-04-25T00:55:47.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f35ec0ae4f0 con 0x7f35ec06f910 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35ec1ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35ec1ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51424/0 (socket says 172.21.15.112:51424) 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 -- 172.21.15.112:0/3658899447 learned_addr learned my addr 172.21.15.112:0/3658899447 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f24ce640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35ec06f910 0x7f35ec09c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f14cc640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35ec06f4d0 0x7f35ec09c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 -- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35ec06f910 msgr2=0x7f35ec09c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35ec06f910 0x7f35ec09c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 -- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35ec06f4d0 msgr2=0x7f35ec09c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.370 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35ec06f4d0 0x7f35ec09c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 -- 172.21.15.112:0/3658899447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec072060 con 0x7f35d80051d0 2024-04-25T00:55:47.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f14cc640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35ec06f4d0 0x7f35ec09c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:47.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f1ccd640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35ec1ce0b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f35e8011840 tx=0x7f35e8011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35e27fc640 1 -- 172.21.15.112:0/3658899447 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8010040 con 0x7f35d80051d0 2024-04-25T00:55:47.371 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35e27fc640 1 -- 172.21.15.112:0/3658899447 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e801a070 con 0x7f35d80051d0 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35e27fc640 1 -- 172.21.15.112:0/3658899447 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e800fd60 con 0x7f35d80051d0 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec09d340 con 0x7f35d80051d0 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec1e0470 con 0x7f35d80051d0 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.360+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec06d800 con 0x7f35d80051d0 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.364+0000 7f35e27fc640 1 -- 172.21.15.112:0/3658899447 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f35e801f460 con 0x7f35d80051d0 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.364+0000 7f35e27fc640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f35d00678c0 0x7f35d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.364+0000 7f35e27fc640 1 -- 172.21.15.112:0/3658899447 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f35e8088cc0 con 0x7f35d80051d0 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.364+0000 7f35f14cc640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f35d00678c0 0x7f35d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.364+0000 7f35f14cc640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f35d00678c0 0x7f35d0069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35ec1424a0 tx=0x7f35dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.368+0000 7f35e27fc640 1 -- 172.21.15.112:0/3658899447 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f35e8059890 con 0x7f35d80051d0 2024-04-25T00:55:47.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.528+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35ec142fd0 con 0x7f35d80051d0 2024-04-25T00:55:47.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.528+0000 7f35e27fc640 1 -- 172.21.15.112:0/3658899447 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f35e8059190 con 0x7f35d80051d0 2024-04-25T00:55:47.531 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:47.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f35d00678c0 msgr2=0x7f35d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f35d00678c0 0x7f35d0069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f35ec1424a0 tx=0x7f35dc009040 comp rx=0 tx=0).stop 2024-04-25T00:55:47.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 msgr2=0x7f35ec1ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35ec1ce0b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f35e8011840 tx=0x7f35e8011d00 comp rx=0 tx=0).stop 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 shutdown_connections 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f35ec06f910 0x7f35ec09c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f35d00678c0 0x7f35d0069d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f35ec06f4d0 0x7f35ec09c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 --2- 172.21.15.112:0/3658899447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35d80051d0 0x7f35ec1ce0b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 >> 172.21.15.112:0/3658899447 conn(0x7f35ec0b8c80 msgr2=0x7f35ec141080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 shutdown_connections 2024-04-25T00:55:47.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.532+0000 7f35f3f31640 1 -- 172.21.15.112:0/3658899447 wait complete. 2024-04-25T00:55:47.551 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:55:47.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f6a74640 1 Processor -- start 2024-04-25T00:55:47.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f6a74640 1 -- start start 2024-04-25T00:55:47.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f6a74640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f37f00b0390 0x7f37f0148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f6a74640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f37f0072b20 0x7f37f0148d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f6a74640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f00b1390 con 0x7f37f00affb0 2024-04-25T00:55:47.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f6a74640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f37f00b1d70 con 0x7f37f0072b20 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f6a74640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f37f00b1600 con 0x7f37f00b0390 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f5011640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f37f0072b20 0x7f37f0148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f4810640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f37f00affb0 0x7f37f009da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57334/0 (socket says 172.21.15.112:57334) 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f4810640 1 -- 172.21.15.112:0/2655006943 learned_addr learned my addr 172.21.15.112:0/2655006943 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37effff640 1 --2- 172.21.15.112:0/2655006943 >> v2:172.21.15.187:3300/0 conn(0x7f37f00b0390 0x7f37f0148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37ef7fe640 1 -- 172.21.15.112:0/2655006943 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3417904013 0 0) 0x7f37f00b1390 con 0x7f37f00affb0 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37ef7fe640 1 -- 172.21.15.112:0/2655006943 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37dc003530 con 0x7f37f00affb0 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f5011640 1 -- 172.21.15.112:0/2655006943 >> v2:172.21.15.187:3300/0 conn(0x7f37f00b0390 msgr2=0x7f37f0148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f5011640 1 --2- 172.21.15.112:0/2655006943 >> v2:172.21.15.187:3300/0 conn(0x7f37f00b0390 0x7f37f0148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f5011640 1 -- 172.21.15.112:0/2655006943 >> v1:172.21.15.112:6789/0 conn(0x7f37f00affb0 legacy=0x7f37f009da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.672+0000 7f37f5011640 1 -- 172.21.15.112:0/2655006943 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f00b1fa0 con 0x7f37f0072b20 2024-04-25T00:55:47.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 --2- 172.21.15.112:0/2655006943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f0072b20 0x7f37f0148d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f37f00b07c0 tx=0x7f37e801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d468f8cb212fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37ef7fe640 1 -- 172.21.15.112:0/2655006943 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80b2070 con 0x7f37f0072b20 2024-04-25T00:55:47.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37ef7fe640 1 -- 172.21.15.112:0/2655006943 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e80a5960 con 0x7f37f0072b20 2024-04-25T00:55:47.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37ef7fe640 1 -- 172.21.15.112:0/2655006943 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80a5d30 con 0x7f37f0072b20 2024-04-25T00:55:47.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- 172.21.15.112:0/2655006943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f0072b20 msgr2=0x7f37f0148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 --2- 172.21.15.112:0/2655006943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f0072b20 0x7f37f0148d40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f37f00b07c0 tx=0x7f37e801c8a0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- 172.21.15.112:0/2655006943 shutdown_connections 2024-04-25T00:55:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 --2- 172.21.15.112:0/2655006943 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f0072b20 0x7f37f0148d40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 --2- 172.21.15.112:0/2655006943 >> v2:172.21.15.187:3300/0 conn(0x7f37f00b0390 0x7f37f0148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- 172.21.15.112:0/2655006943 >> 172.21.15.112:0/2655006943 conn(0x7f37f00b8c80 msgr2=0x7f37f00b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- 172.21.15.112:0/2655006943 shutdown_connections 2024-04-25T00:55:47.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- 172.21.15.112:0/2655006943 wait complete. 2024-04-25T00:55:47.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 Processor -- start 2024-04-25T00:55:47.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- start start 2024-04-25T00:55:47.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37f0072b20 0x7f37f009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f00affb0 0x7f37f00aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f00b0390 0x7f37f00aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0152290 con 0x7f37f00b0390 2024-04-25T00:55:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f37f009e170 con 0x7f37f00affb0 2024-04-25T00:55:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f37f0152410 con 0x7f37f0072b20 2024-04-25T00:55:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f00b0390 0x7f37f00aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f00b0390 0x7f37f00aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51438/0 (socket says 172.21.15.112:51438) 2024-04-25T00:55:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 -- 172.21.15.112:0/609684947 learned_addr learned my addr 172.21.15.112:0/609684947 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:47.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f4810640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37f0072b20 0x7f37f009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37effff640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f00affb0 0x7f37f00aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 -- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37f0072b20 msgr2=0x7f37f009ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37f0072b20 0x7f37f009ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 -- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f00affb0 msgr2=0x7f37f00aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f00affb0 0x7f37f00aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 -- 172.21.15.112:0/609684947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f01e0d10 con 0x7f37f00b0390 2024-04-25T00:55:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f4810640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37f0072b20 0x7f37f009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:47.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f5011640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f00b0390 0x7f37f00aef80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f37e80b0040 tx=0x7f37e8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37ed7fa640 1 -- 172.21.15.112:0/609684947 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80b2050 con 0x7f37f00b0390 2024-04-25T00:55:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37ed7fa640 1 -- 172.21.15.112:0/609684947 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e80be070 con 0x7f37f00b0390 2024-04-25T00:55:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f01e06c0 con 0x7f37f00b0390 2024-04-25T00:55:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37ed7fa640 1 -- 172.21.15.112:0/609684947 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80adc40 con 0x7f37f00b0390 2024-04-25T00:55:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.676+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f01e02b0 con 0x7f37f00b0390 2024-04-25T00:55:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.680+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f014f390 con 0x7f37f00b0390 2024-04-25T00:55:47.688 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.680+0000 7f37ed7fa640 1 -- 172.21.15.112:0/609684947 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f37e80c3460 con 0x7f37f00b0390 2024-04-25T00:55:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.680+0000 7f37ed7fa640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f37d00678c0 0x7f37d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.680+0000 7f37ed7fa640 1 -- 172.21.15.112:0/609684947 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f37e8120e20 con 0x7f37f00b0390 2024-04-25T00:55:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.680+0000 7f37f4810640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f37d00678c0 0x7f37d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.680+0000 7f37f4810640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f37d00678c0 0x7f37d0069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005890 tx=0x7f37d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.684+0000 7f37ed7fa640 1 -- 172.21.15.112:0/609684947 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f37e8016520 con 0x7f37f00b0390 2024-04-25T00:55:47.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.832+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f37f00ae7a0 con 0x7f37f00b0390 2024-04-25T00:55:47.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.832+0000 7f37ed7fa640 1 -- 172.21.15.112:0/609684947 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f37e80f19f0 con 0x7f37f00b0390 2024-04-25T00:55:47.836 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f37d00678c0 msgr2=0x7f37d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f37d00678c0 0x7f37d0069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f37d8005890 tx=0x7f37d8009040 comp rx=0 tx=0).stop 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f00b0390 msgr2=0x7f37f00aef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f00b0390 0x7f37f00aef80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f37e80b0040 tx=0x7f37e8019640 comp rx=0 tx=0).stop 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 shutdown_connections 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37f00b0390 0x7f37f00aef80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f37d00678c0 0x7f37d0069d80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f37f00affb0 0x7f37f00aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 --2- 172.21.15.112:0/609684947 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f37f0072b20 0x7f37f009ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 >> 172.21.15.112:0/609684947 conn(0x7f37f00b8c80 msgr2=0x7f37f01462b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 shutdown_connections 2024-04-25T00:55:47.839 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.836+0000 7f37f6a74640 1 -- 172.21.15.112:0/609684947 wait complete. 2024-04-25T00:55:47.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:55:47.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f06074cf640 1 Processor -- start 2024-04-25T00:55:47.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f06074cf640 1 -- start start 2024-04-25T00:55:47.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f06074cf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f06074cf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0600072b20 0x7f0600148d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f06074cf640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06000b1390 con 0x7f06000affb0 2024-04-25T00:55:47.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f06074cf640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f06000b1d70 con 0x7f0600072b20 2024-04-25T00:55:47.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f06074cf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f06000b1600 con 0x7f06000b0390 2024-04-25T00:55:47.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0605a6c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0600072b20 0x7f0600148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0605a6c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0600072b20 0x7f0600148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51454/0 (socket says 172.21.15.112:51454) 2024-04-25T00:55:47.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0605a6c640 1 -- 172.21.15.112:0/145269808 learned_addr learned my addr 172.21.15.112:0/145269808 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:47.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0604a6a640 1 --2- 172.21.15.112:0/145269808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0605a6c640 1 -- 172.21.15.112:0/145269808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 msgr2=0x7f0600148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0605a6c640 1 --2- 172.21.15.112:0/145269808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0605a6c640 1 -- 172.21.15.112:0/145269808 >> v1:172.21.15.187:6789/0 conn(0x7f06000affb0 legacy=0x7f060009da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:47.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0605a6c640 1 -- 172.21.15.112:0/145269808 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06000b1fa0 con 0x7f0600072b20 2024-04-25T00:55:47.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.960+0000 7f0604a6a640 1 --2- 172.21.15.112:0/145269808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f0605a6c640 1 --2- 172.21.15.112:0/145269808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f0600148d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009e80 tx=0x7f05f801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271f0de9b3192d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f05f7fff640 1 -- 172.21.15.112:0/145269808 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f80a57c0 con 0x7f0600072b20 2024-04-25T00:55:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f05f7fff640 1 -- 172.21.15.112:0/145269808 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f80a5960 con 0x7f0600072b20 2024-04-25T00:55:47.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f05f7fff640 1 -- 172.21.15.112:0/145269808 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f80a5d30 con 0x7f0600072b20 2024-04-25T00:55:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- 172.21.15.112:0/145269808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 msgr2=0x7f0600148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 --2- 172.21.15.112:0/145269808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f0600148d40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009e80 tx=0x7f05f801c940 comp rx=0 tx=0).stop 2024-04-25T00:55:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- 172.21.15.112:0/145269808 shutdown_connections 2024-04-25T00:55:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 --2- 172.21.15.112:0/145269808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f0600148d40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 --2- 172.21.15.112:0/145269808 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.968 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- 172.21.15.112:0/145269808 >> 172.21.15.112:0/145269808 conn(0x7f06000b8c80 msgr2=0x7f06000b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- 172.21.15.112:0/145269808 shutdown_connections 2024-04-25T00:55:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- 172.21.15.112:0/145269808 wait complete. 2024-04-25T00:55:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 Processor -- start 2024-04-25T00:55:47.969 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- start start 2024-04-25T00:55:47.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f06001e18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06000affb0 0x7f0600143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600138810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0600152290 con 0x7f0600072b20 2024-04-25T00:55:47.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f060009e170 con 0x7f06000affb0 2024-04-25T00:55:47.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0600152410 con 0x7f06000b0390 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f06001e18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f06001e18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51460/0 (socket says 172.21.15.112:51460) 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 -- 172.21.15.112:0/1599812575 learned_addr learned my addr 172.21.15.112:0/1599812575 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f0605a6c640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600138810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 -- 172.21.15.112:0/1599812575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 msgr2=0x7f0600138810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600138810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f0604a6a640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06000affb0 0x7f0600143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 -- 172.21.15.112:0/1599812575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06000affb0 msgr2=0x7f0600143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:47.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06000affb0 0x7f0600143540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 -- 172.21.15.112:0/1599812575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060013fdc0 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f060526b640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f06001e18f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c450 tx=0x7f05fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f05f5ffb640 1 -- 172.21.15.112:0/1599812575 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc012830 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f05f5ffb640 1 -- 172.21.15.112:0/1599812575 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc017070 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060009d2b0 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f05f5ffb640 1 -- 172.21.15.112:0/1599812575 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc012c30 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.964+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0600143a80 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.968+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06000b1040 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.968+0000 7f05f5ffb640 1 -- 172.21.15.112:0/1599812575 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f05fc01c460 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.968+0000 7f05f5ffb640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f05e40678f0 0x7f05e4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.968+0000 7f05f5ffb640 1 -- 172.21.15.112:0/1599812575 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f05fc085a60 con 0x7f0600072b20 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.968+0000 7f0604a6a640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f05e40678f0 0x7f05e4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:47.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.968+0000 7f0604a6a640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f05e40678f0 0x7f05e4069db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f05f0006fd0 tx=0x7f05f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:47.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:47.972+0000 7f05f5ffb640 1 -- 172.21.15.112:0/1599812575 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f05fc0565a0 con 0x7f0600072b20 2024-04-25T00:55:48.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.116+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f060004f5f0 con 0x7f0600072b20 2024-04-25T00:55:48.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.116+0000 7f05f5ffb640 1 -- 172.21.15.112:0/1599812575 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f05fc055ea0 con 0x7f0600072b20 2024-04-25T00:55:48.121 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:55:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f05e40678f0 msgr2=0x7f05e4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f05e40678f0 0x7f05e4069db0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f05f0006fd0 tx=0x7f05f0008040 comp rx=0 tx=0).stop 2024-04-25T00:55:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 msgr2=0x7f06001e18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f06001e18f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00c450 tx=0x7f05fc00c910 comp rx=0 tx=0).stop 2024-04-25T00:55:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 shutdown_connections 2024-04-25T00:55:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f06000b0390 0x7f0600138810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f05e40678f0 0x7f05e4069db0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f06000affb0 0x7f0600143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 --2- 172.21.15.112:0/1599812575 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0600072b20 0x7f06001e18f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 >> 172.21.15.112:0/1599812575 conn(0x7f06000b8c80 msgr2=0x7f0600146740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 shutdown_connections 2024-04-25T00:55:48.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.120+0000 7f06074cf640 1 -- 172.21.15.112:0/1599812575 wait complete. 2024-04-25T00:55:48.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:55:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.252+0000 7f2da9c31640 1 Processor -- start 2024-04-25T00:55:48.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.252+0000 7f2da9c31640 1 -- start start 2024-04-25T00:55:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da9c31640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2da40b23f0 0x7f2da40add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da9c31640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2da40b3ce0 0x7f2da414fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da9c31640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2da40aeec0 con 0x7f2da4145200 2024-04-25T00:55:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da9c31640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f2da40af780 con 0x7f2da40b3ce0 2024-04-25T00:55:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da9c31640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2da40b27d0 con 0x7f2da40b23f0 2024-04-25T00:55:48.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2da40b3ce0 0x7f2da414fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da3fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2da40b3ce0 0x7f2da414fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51474/0 (socket says 172.21.15.112:51474) 2024-04-25T00:55:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da3fff640 1 -- 172.21.15.112:0/532665616 learned_addr learned my addr 172.21.15.112:0/532665616 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da37fe640 1 --2- 172.21.15.112:0/532665616 >> v2:172.21.15.187:3300/0 conn(0x7f2da40b23f0 0x7f2da40add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da3fff640 1 -- 172.21.15.112:0/532665616 >> v2:172.21.15.187:3300/0 conn(0x7f2da40b23f0 msgr2=0x7f2da40add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da3fff640 1 --2- 172.21.15.112:0/532665616 >> v2:172.21.15.187:3300/0 conn(0x7f2da40b23f0 0x7f2da40add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da3fff640 1 -- 172.21.15.112:0/532665616 >> v1:172.21.15.187:6789/0 conn(0x7f2da4145200 legacy=0x7f2da4072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:48.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.256+0000 7f2da3fff640 1 -- 172.21.15.112:0/532665616 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da40afa20 con 0x7f2da40b3ce0 2024-04-25T00:55:48.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da3fff640 1 --2- 172.21.15.112:0/532665616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b3ce0 0x7f2da414fb40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2da406d9f0 tx=0x7f2d9c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f3b2fde30a1efa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da27fc640 1 -- 172.21.15.112:0/532665616 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0a5810 con 0x7f2da40b3ce0 2024-04-25T00:55:48.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da27fc640 1 -- 172.21.15.112:0/532665616 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d9c0a59b0 con 0x7f2da40b3ce0 2024-04-25T00:55:48.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da27fc640 1 -- 172.21.15.112:0/532665616 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c0a5d80 con 0x7f2da40b3ce0 2024-04-25T00:55:48.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- 172.21.15.112:0/532665616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b3ce0 msgr2=0x7f2da414fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 --2- 172.21.15.112:0/532665616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b3ce0 0x7f2da414fb40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2da406d9f0 tx=0x7f2d9c01c9b0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- 172.21.15.112:0/532665616 shutdown_connections 2024-04-25T00:55:48.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 --2- 172.21.15.112:0/532665616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b3ce0 0x7f2da414fb40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 --2- 172.21.15.112:0/532665616 >> v2:172.21.15.187:3300/0 conn(0x7f2da40b23f0 0x7f2da40add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- 172.21.15.112:0/532665616 >> 172.21.15.112:0/532665616 conn(0x7f2da40b5980 msgr2=0x7f2da40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:48.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- 172.21.15.112:0/532665616 shutdown_connections 2024-04-25T00:55:48.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- 172.21.15.112:0/532665616 wait complete. 2024-04-25T00:55:48.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 Processor -- start 2024-04-25T00:55:48.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- start start 2024-04-25T00:55:48.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b23f0 0x7f2da41465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40b3ce0 0x7f2da41ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da4145200 0x7f2da41e0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2da40b52d0 con 0x7f2da40b23f0 2024-04-25T00:55:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2da40b5150 con 0x7f2da4145200 2024-04-25T00:55:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da9c31640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2da41528a0 con 0x7f2da40b3ce0 2024-04-25T00:55:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b23f0 0x7f2da41465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b23f0 0x7f2da41465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51490/0 (socket says 172.21.15.112:51490) 2024-04-25T00:55:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 -- 172.21.15.112:0/1860287602 learned_addr learned my addr 172.21.15.112:0/1860287602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:48.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da2ffd640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40b3ce0 0x7f2da41ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da3fff640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da4145200 0x7f2da41e0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 -- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40b3ce0 msgr2=0x7f2da41ddf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40b3ce0 0x7f2da41ddf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 -- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da4145200 msgr2=0x7f2da41e0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da4145200 0x7f2da41e0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 -- 172.21.15.112:0/1860287602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2da41ddaa0 con 0x7f2da40b23f0 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da3fff640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da4145200 0x7f2da41e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da2ffd640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40b3ce0 0x7f2da41ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.260+0000 7f2da37fe640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b23f0 0x7f2da41465e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400d800 tx=0x7f2d9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2d8bfff640 1 -- 172.21.15.112:0/1860287602 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400b6c0 con 0x7f2da40b23f0 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2d8bfff640 1 -- 172.21.15.112:0/1860287602 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d94019070 con 0x7f2da40b23f0 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2da413f4e0 con 0x7f2da40b23f0 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2d8bfff640 1 -- 172.21.15.112:0/1860287602 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d9400bae0 con 0x7f2da40b23f0 2024-04-25T00:55:48.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2da41395c0 con 0x7f2da40b23f0 2024-04-25T00:55:48.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da404f580 con 0x7f2da40b23f0 2024-04-25T00:55:48.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2d8bfff640 1 -- 172.21.15.112:0/1860287602 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2d9400bc80 con 0x7f2da40b23f0 2024-04-25T00:55:48.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2d8bfff640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2d900678c0 0x7f2d90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2d8bfff640 1 -- 172.21.15.112:0/1860287602 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2d94086140 con 0x7f2da40b23f0 2024-04-25T00:55:48.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2da2ffd640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2d900678c0 0x7f2d90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.264+0000 7f2da2ffd640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2d900678c0 0x7f2d90069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2da413f0c0 tx=0x7f2d98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.268+0000 7f2d8bfff640 1 -- 172.21.15.112:0/1860287602 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2d94056d10 con 0x7f2da40b23f0 2024-04-25T00:55:48.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.460+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2da4139850 con 0x7f2da40b23f0 2024-04-25T00:55:48.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2d8bfff640 1 -- 172.21.15.112:0/1860287602 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f2d94056610 con 0x7f2da40b23f0 2024-04-25T00:55:48.467 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:48.468 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:55:38.966915+0000","modified":"2024-04-25T00:55:42.002877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:55:48.468 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T00:55:48.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2d900678c0 msgr2=0x7f2d90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2d900678c0 0x7f2d90069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2da413f0c0 tx=0x7f2d98008040 comp rx=0 tx=0).stop 2024-04-25T00:55:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b23f0 msgr2=0x7f2da41465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b23f0 0x7f2da41465e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2d9400d800 tx=0x7f2d9400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 shutdown_connections 2024-04-25T00:55:48.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2d900678c0 0x7f2d90069d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2da4145200 0x7f2da41e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2da40b3ce0 0x7f2da41ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 --2- 172.21.15.112:0/1860287602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2da40b23f0 0x7f2da41465e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 >> 172.21.15.112:0/1860287602 conn(0x7f2da40b5980 msgr2=0x7f2da406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 shutdown_connections 2024-04-25T00:55:48.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.464+0000 7f2da9c31640 1 -- 172.21.15.112:0/1860287602 wait complete. 2024-04-25T00:55:48.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:48.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.600+0000 7f593f4bd640 1 Processor -- start 2024-04-25T00:55:48.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.600+0000 7f593f4bd640 1 -- start start 2024-04-25T00:55:48.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593f4bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f593809c270 0x7f5938147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593f4bd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5938147650 0x7f5938148240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593f4bd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59380bbec0 con 0x7f59380722d0 2024-04-25T00:55:48.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593f4bd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f59380baaf0 con 0x7f5938147650 2024-04-25T00:55:48.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593f4bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f59380bc5c0 con 0x7f593809c270 2024-04-25T00:55:48.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593ca58640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f593809c270 0x7f5938147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593da5a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5938147650 0x7f5938148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593ca58640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f593809c270 0x7f5938147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51614/0 (socket says 172.21.15.112:51614) 2024-04-25T00:55:48.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593ca58640 1 -- 172.21.15.112:0/1394855038 learned_addr learned my addr 172.21.15.112:0/1394855038 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:48.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593da5a640 1 -- 172.21.15.112:0/1394855038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f593809c270 msgr2=0x7f5938147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:48.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593da5a640 1 --2- 172.21.15.112:0/1394855038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f593809c270 0x7f5938147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593da5a640 1 -- 172.21.15.112:0/1394855038 >> v1:172.21.15.187:6789/0 conn(0x7f59380722d0 legacy=0x7f5938072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.604+0000 7f593da5a640 1 -- 172.21.15.112:0/1394855038 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380bc8a0 con 0x7f5938147650 2024-04-25T00:55:48.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593da5a640 1 --2- 172.21.15.112:0/1394855038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5938147650 0x7f5938148240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5928009d90 tx=0x7f592801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8986779e901707ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f5937fff640 1 -- 172.21.15.112:0/1394855038 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59280b2070 con 0x7f5938147650 2024-04-25T00:55:48.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f5937fff640 1 -- 172.21.15.112:0/1394855038 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59280a5a00 con 0x7f5938147650 2024-04-25T00:55:48.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f5937fff640 1 -- 172.21.15.112:0/1394855038 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59280a5dd0 con 0x7f5938147650 2024-04-25T00:55:48.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- 172.21.15.112:0/1394855038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5938147650 msgr2=0x7f5938148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 --2- 172.21.15.112:0/1394855038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5938147650 0x7f5938148240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5928009d90 tx=0x7f592801c940 comp rx=0 tx=0).stop 2024-04-25T00:55:48.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- 172.21.15.112:0/1394855038 shutdown_connections 2024-04-25T00:55:48.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 --2- 172.21.15.112:0/1394855038 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5938147650 0x7f5938148240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 --2- 172.21.15.112:0/1394855038 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f593809c270 0x7f5938147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- 172.21.15.112:0/1394855038 >> 172.21.15.112:0/1394855038 conn(0x7f593809e8c0 msgr2=0x7f593809ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:48.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- 172.21.15.112:0/1394855038 shutdown_connections 2024-04-25T00:55:48.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- 172.21.15.112:0/1394855038 wait complete. 2024-04-25T00:55:48.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 Processor -- start 2024-04-25T00:55:48.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- start start 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380722d0 0x7f59380b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f593809c270 0x7f59381e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5938147650 0x7f5938144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5938154890 con 0x7f59380722d0 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5938154710 con 0x7f593809c270 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593f4bd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5938154a10 con 0x7f5938147650 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593d259640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380722d0 0x7f59380b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593d259640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380722d0 0x7f59380b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51504/0 (socket says 172.21.15.112:51504) 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593d259640 1 -- 172.21.15.112:0/3490366821 learned_addr learned my addr 172.21.15.112:0/3490366821 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593da5a640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5938147650 0x7f5938144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593ca58640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f593809c270 0x7f59381e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593d259640 1 -- 172.21.15.112:0/3490366821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5938147650 msgr2=0x7f5938144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593d259640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5938147650 0x7f5938144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593d259640 1 -- 172.21.15.112:0/3490366821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f593809c270 msgr2=0x7f59381e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593d259640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f593809c270 0x7f59381e34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.608+0000 7f593d259640 1 -- 172.21.15.112:0/3490366821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380b5720 con 0x7f59380722d0 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f593d259640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380722d0 0x7f59380b4460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f593000c370 tx=0x7f593000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f5935ffb640 1 -- 172.21.15.112:0/3490366821 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59300052d0 con 0x7f59380722d0 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f5935ffb640 1 -- 172.21.15.112:0/3490366821 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5930005470 con 0x7f59380722d0 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f5935ffb640 1 -- 172.21.15.112:0/3490366821 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930007260 con 0x7f59380722d0 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59380b5990 con 0x7f59380722d0 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59380b2ef0 con 0x7f59380722d0 2024-04-25T00:55:48.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f593804f580 con 0x7f59380722d0 2024-04-25T00:55:48.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f5935ffb640 1 -- 172.21.15.112:0/3490366821 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5930007400 con 0x7f59380722d0 2024-04-25T00:55:48.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f5935ffb640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59240678f0 0x7f5924069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f593ca58640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59240678f0 0x7f5924069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f5935ffb640 1 -- 172.21.15.112:0/3490366821 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5930085050 con 0x7f59380722d0 2024-04-25T00:55:48.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.612+0000 7f593ca58640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59240678f0 0x7f5924069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59380b0220 tx=0x7f592c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.616+0000 7f5935ffb640 1 -- 172.21.15.112:0/3490366821 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5930055c20 con 0x7f59380722d0 2024-04-25T00:55:48.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.776+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59380b30c0 con 0x7f59380722d0 2024-04-25T00:55:48.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.780+0000 7f5935ffb640 1 -- 172.21.15.112:0/3490366821 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f5930055520 con 0x7f59380722d0 2024-04-25T00:55:48.784 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:48.785 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59240678f0 msgr2=0x7f5924069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59240678f0 0x7f5924069db0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f59380b0220 tx=0x7f592c008040 comp rx=0 tx=0).stop 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380722d0 msgr2=0x7f59380b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380722d0 0x7f59380b4460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f593000c370 tx=0x7f593000c830 comp rx=0 tx=0).stop 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 shutdown_connections 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5938147650 0x7f5938144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59240678f0 0x7f5924069db0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f593809c270 0x7f59381e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 --2- 172.21.15.112:0/3490366821 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f59380722d0 0x7f59380b4460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 >> 172.21.15.112:0/3490366821 conn(0x7f593809e8c0 msgr2=0x7f59380b7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 shutdown_connections 2024-04-25T00:55:48.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.784+0000 7f593f4bd640 1 -- 172.21.15.112:0/3490366821 wait complete. 2024-04-25T00:55:48.804 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:48.804 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f88485c0640 1 Processor -- start 2024-04-25T00:55:48.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f88485c0640 1 -- start start 2024-04-25T00:55:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f88485c0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400add40 0x7f88400b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f88485c0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f884006caf0 0x7f88400b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f88485c0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f884009e2b0 con 0x7f884009ded0 2024-04-25T00:55:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f88485c0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f88400ae790 con 0x7f884006caf0 2024-04-25T00:55:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f88485c0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f88400aea70 con 0x7f88400add40 2024-04-25T00:55:48.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884635c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f884009ded0 0x7f88400b34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57346/0 (socket says 172.21.15.112:57346) 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884635c640 1 -- 172.21.15.112:0/1693709158 learned_addr learned my addr 172.21.15.112:0/1693709158 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291503784 0 0) 0x7f884009e2b0 con 0x7f884009ded0 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f882c003450 con 0x7f884009ded0 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8845b5b640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400add40 0x7f88400b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8846b5d640 1 --2- 172.21.15.112:0/1693709158 >> v2:172.21.15.187:3300/0 conn(0x7f884006caf0 0x7f88400b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2476083963 0 0) 0x7f882c003450 con 0x7f884009ded0 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400add40 msgr2=0x7f88400b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400add40 0x7f88400b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 >> v2:172.21.15.187:3300/0 conn(0x7f884006caf0 msgr2=0x7f88400b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 --2- 172.21.15.112:0/1693709158 >> v2:172.21.15.187:3300/0 conn(0x7f884006caf0 0x7f88400b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88400aede0 con 0x7f884009ded0 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f883c001a00 con 0x7f884009ded0 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 >> v1:172.21.15.112:6789/0 conn(0x7f884009ded0 legacy=0x7f88400b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f882c003ab0 0x7f882c003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400add40 0x7f882c00a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c00b030 0x7f882c00d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88400ae790 con 0x7f882c00b030 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88400aea70 con 0x7f882c003ab0 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f882c00ddf0 con 0x7f88400add40 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8846b5d640 1 --2- 172.21.15.112:0/1693709158 >> v2:172.21.15.187:3300/0 conn(0x7f884006caf0 0x7f88400b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8846b5d640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c00b030 0x7f882c00d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8845b5b640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400add40 0x7f882c00a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884635c640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f882c003ab0 0x7f882c003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8846b5d640 1 -- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400add40 msgr2=0x7f882c00a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8846b5d640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400add40 0x7f882c00a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8846b5d640 1 -- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f882c003ab0 msgr2=0x7f882c003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8846b5d640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f882c003ab0 0x7f882c003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f8846b5d640 1 -- 172.21.15.112:0/1693709158 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c003450 con 0x7f882c00b030 2024-04-25T00:55:48.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.912+0000 7f884635c640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f882c003ab0 0x7f882c003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c00b030 0x7f882c00d4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88380098b0 tx=0x7f883801c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684593573e83cf59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883800f040 con 0x7f882c00b030 2024-04-25T00:55:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8838004020 con 0x7f882c00b030 2024-04-25T00:55:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f884535a640 1 -- 172.21.15.112:0/1693709158 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88380043f0 con 0x7f882c00b030 2024-04-25T00:55:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- 172.21.15.112:0/1693709158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c00b030 msgr2=0x7f882c00d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 --2- 172.21.15.112:0/1693709158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f882c00b030 0x7f882c00d4c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f88380098b0 tx=0x7f883801c630 comp rx=0 tx=0).stop 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 -- 172.21.15.112:0/1693709158 reap_dead start 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- 172.21.15.112:0/1693709158 shutdown_connections 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- 172.21.15.112:0/1693709158 >> 172.21.15.112:0/1693709158 conn(0x7f88400b8c80 msgr2=0x7f88400b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- 172.21.15.112:0/1693709158 shutdown_connections 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- 172.21.15.112:0/1693709158 wait complete. 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 Processor -- start 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- start start 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400b8210 0x7f88401dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884009ae80 0x7f884013cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f884013d0e0 0x7f88400b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f88400b4280 con 0x7f884013d0e0 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f88400b4100 con 0x7f884009ae80 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f884014e4e0 con 0x7f88400b8210 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f884013d0e0 0x7f88400b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f884013d0e0 0x7f88400b1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51518/0 (socket says 172.21.15.112:51518) 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 -- 172.21.15.112:0/2634772639 learned_addr learned my addr 172.21.15.112:0/2634772639 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8845b5b640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884009ae80 0x7f884013cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f884635c640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400b8210 0x7f88401dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 -- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400b8210 msgr2=0x7f88401dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400b8210 0x7f88401dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 -- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884009ae80 msgr2=0x7f884013cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884009ae80 0x7f884013cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 -- 172.21.15.112:0/2634772639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88400b85f0 con 0x7f884013d0e0 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f884635c640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400b8210 0x7f88401dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8846b5d640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f884013d0e0 0x7f88400b1490 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f883801cb10 tx=0x7f882c003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8836ffd640 1 -- 172.21.15.112:0/2634772639 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883800f000 con 0x7f884013d0e0 2024-04-25T00:55:48.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8836ffd640 1 -- 172.21.15.112:0/2634772639 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88380bd070 con 0x7f884013d0e0 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884009d360 con 0x7f884013d0e0 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f8836ffd640 1 -- 172.21.15.112:0/2634772639 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838003820 con 0x7f884013d0e0 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.916+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884009c0b0 con 0x7f884013d0e0 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.920+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884006d800 con 0x7f884013d0e0 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.920+0000 7f8836ffd640 1 -- 172.21.15.112:0/2634772639 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f88380b1070 con 0x7f884013d0e0 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.920+0000 7f8836ffd640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f88240678c0 0x7f8824069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.920+0000 7f8836ffd640 1 -- 172.21.15.112:0/2634772639 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f883811fb70 con 0x7f884013d0e0 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.920+0000 7f884635c640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f88240678c0 0x7f8824069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:48.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.920+0000 7f884635c640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f88240678c0 0x7f8824069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f883c001590 tx=0x7f883c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:48.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:48.924+0000 7f8836ffd640 1 -- 172.21.15.112:0/2634772639 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f88380a7070 con 0x7f884013d0e0 2024-04-25T00:55:49.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.072+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f884004f5f0 con 0x7f884013d0e0 2024-04-25T00:55:49.077 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.072+0000 7f8836ffd640 1 -- 172.21.15.112:0/2634772639 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8838016340 con 0x7f884013d0e0 2024-04-25T00:55:49.078 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:49.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f88240678c0 msgr2=0x7f8824069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f88240678c0 0x7f8824069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f883c001590 tx=0x7f883c009040 comp rx=0 tx=0).stop 2024-04-25T00:55:49.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f884013d0e0 msgr2=0x7f88400b1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f884013d0e0 0x7f88400b1490 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f883801cb10 tx=0x7f882c003740 comp rx=0 tx=0).stop 2024-04-25T00:55:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 shutdown_connections 2024-04-25T00:55:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f884013d0e0 0x7f88400b1490 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f88240678c0 0x7f8824069d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f884009ae80 0x7f884013cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 --2- 172.21.15.112:0/2634772639 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f88400b8210 0x7f88401dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 >> 172.21.15.112:0/2634772639 conn(0x7f88400b8c80 msgr2=0x7f884013fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 shutdown_connections 2024-04-25T00:55:49.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.076+0000 7f88485c0640 1 -- 172.21.15.112:0/2634772639 wait complete. 2024-04-25T00:55:49.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:55:49.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.200+0000 7fadfe5be640 1 Processor -- start 2024-04-25T00:55:49.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.200+0000 7fadfe5be640 1 -- start start 2024-04-25T00:55:49.206 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadfe5be640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fadf8147900 0x7fadf8147d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadfe5be640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf814ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadfe5be640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf80af4e0 con 0x7fadf806c170 2024-04-25T00:55:49.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadfe5be640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fadf80ae8a0 con 0x7fadf8147900 2024-04-25T00:55:49.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadfe5be640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadf8148710 con 0x7fadf81482a0 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadfcb5b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf814ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadeffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fadf8147900 0x7fadf8147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadfcb5b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf814ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:51660/0 (socket says 172.21.15.112:51660) 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadeffff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fadf8147900 0x7fadf8147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41280/0 (socket says 172.21.15.112:41280) 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadfcb5b640 1 -- 172.21.15.112:0/19104834 learned_addr learned my addr 172.21.15.112:0/19104834 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadeffff640 1 -- 172.21.15.112:0/19104834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 msgr2=0x7fadf814ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadeffff640 1 --2- 172.21.15.112:0/19104834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf814ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadeffff640 1 -- 172.21.15.112:0/19104834 >> v1:172.21.15.187:6789/0 conn(0x7fadf806c170 legacy=0x7fadf80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadeffff640 1 -- 172.21.15.112:0/19104834 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf806c550 con 0x7fadf8147900 2024-04-25T00:55:49.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.204+0000 7fadeffff640 1 --2- 172.21.15.112:0/19104834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 0x7fadf8147d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fade80097c0 tx=0x7fade801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a58785c732a43781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf77fe640 1 -- 172.21.15.112:0/19104834 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80b0070 con 0x7fadf8147900 2024-04-25T00:55:49.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf77fe640 1 -- 172.21.15.112:0/19104834 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade80ab600 con 0x7fadf8147900 2024-04-25T00:55:49.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf77fe640 1 -- 172.21.15.112:0/19104834 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80ab9d0 con 0x7fadf8147900 2024-04-25T00:55:49.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- 172.21.15.112:0/19104834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 msgr2=0x7fadf8147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 --2- 172.21.15.112:0/19104834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 0x7fadf8147d60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fade80097c0 tx=0x7fade801c410 comp rx=0 tx=0).stop 2024-04-25T00:55:49.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- 172.21.15.112:0/19104834 shutdown_connections 2024-04-25T00:55:49.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 --2- 172.21.15.112:0/19104834 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf814ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 --2- 172.21.15.112:0/19104834 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 0x7fadf8147d60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- 172.21.15.112:0/19104834 >> 172.21.15.112:0/19104834 conn(0x7fadf80b8c80 msgr2=0x7fadf80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:49.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- 172.21.15.112:0/19104834 shutdown_connections 2024-04-25T00:55:49.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- 172.21.15.112:0/19104834 wait complete. 2024-04-25T00:55:49.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 Processor -- start 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- start start 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf806c170 0x7fadf809d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 0x7fadf81e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf806c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fadf80adef0 con 0x7fadf806c170 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fadf80ae530 con 0x7fadf8147900 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fadf80b2e80 con 0x7fadf81482a0 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf806c170 0x7fadf809d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf806c170 0x7fadf809d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51526/0 (socket says 172.21.15.112:51526) 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 -- 172.21.15.112:0/1013668160 learned_addr learned my addr 172.21.15.112:0/1013668160 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfcb5b640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf806c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadeffff640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 0x7fadf81e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 -- 172.21.15.112:0/1013668160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 msgr2=0x7fadf806c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf806c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 -- 172.21.15.112:0/1013668160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 msgr2=0x7fadf81e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 0x7fadf81e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 -- 172.21.15.112:0/1013668160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf8071600 con 0x7fadf806c170 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf7fff640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf806c170 0x7fadf809d410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fade000c500 tx=0x7fade000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf57fa640 1 -- 172.21.15.112:0/1013668160 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0016020 con 0x7fadf806c170 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf57fa640 1 -- 172.21.15.112:0/1013668160 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade0005220 con 0x7fadf806c170 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf80718e0 con 0x7fadf806c170 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadf57fa640 1 -- 172.21.15.112:0/1013668160 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0007030 con 0x7fadf806c170 2024-04-25T00:55:49.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.208+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf80712d0 con 0x7fadf806c170 2024-04-25T00:55:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.212+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf8071c10 con 0x7fadf806c170 2024-04-25T00:55:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.212+0000 7fadf57fa640 1 -- 172.21.15.112:0/1013668160 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fade00071f0 con 0x7fadf806c170 2024-04-25T00:55:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.212+0000 7fadf57fa640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fadd80678c0 0x7fadd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.212+0000 7fadeffff640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fadd80678c0 0x7fadd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.212+0000 7fadf57fa640 1 -- 172.21.15.112:0/1013668160 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fade0085660 con 0x7fadf806c170 2024-04-25T00:55:49.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.212+0000 7fadeffff640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fadd80678c0 0x7fadd8069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fade80ad000 tx=0x7fade8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.216+0000 7fadf57fa640 1 -- 172.21.15.112:0/1013668160 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fade0020020 con 0x7fadf806c170 2024-04-25T00:55:49.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fadf804f5f0 con 0x7fadf806c170 2024-04-25T00:55:49.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadf57fa640 1 -- 172.21.15.112:0/1013668160 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fade00561b0 con 0x7fadf806c170 2024-04-25T00:55:49.379 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:55:49.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fadd80678c0 msgr2=0x7fadd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fadd80678c0 0x7fadd8069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fade80ad000 tx=0x7fade8008040 comp rx=0 tx=0).stop 2024-04-25T00:55:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf806c170 msgr2=0x7fadf809d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf806c170 0x7fadf809d410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fade000c500 tx=0x7fade000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 shutdown_connections 2024-04-25T00:55:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fadf81482a0 0x7fadf806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fadd80678c0 0x7fadd8069d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.382 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fadf8147900 0x7fadf81e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 --2- 172.21.15.112:0/1013668160 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fadf806c170 0x7fadf809d410 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 >> 172.21.15.112:0/1013668160 conn(0x7fadf80b8c80 msgr2=0x7fadf8142ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:49.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 shutdown_connections 2024-04-25T00:55:49.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.376+0000 7fadfe5be640 1 -- 172.21.15.112:0/1013668160 wait complete. 2024-04-25T00:55:49.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:55:49.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce107c3640 1 Processor -- start 2024-04-25T00:55:49.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce107c3640 1 -- start start 2024-04-25T00:55:49.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce107c3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fce0814b2b0 0x7fce08140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce107c3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce080aea40 con 0x7fce08074550 2024-04-25T00:55:49.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce107c3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce080aed20 con 0x7fce080ade90 2024-04-25T00:55:49.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce107c3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fce080af090 con 0x7fce0814b2b0 2024-04-25T00:55:49.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce0e55f640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fce08074550 0x7fce0814aba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57348/0 (socket says 172.21.15.112:57348) 2024-04-25T00:55:49.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce0e55f640 1 -- 172.21.15.112:0/2973846491 learned_addr learned my addr 172.21.15.112:0/2973846491 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:49.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce0dd5e640 1 --2- 172.21.15.112:0/2973846491 >> v2:172.21.15.112:3300/0 conn(0x7fce0814b2b0 0x7fce08140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce0d55d640 1 -- 172.21.15.112:0/2973846491 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1808574139 0 0) 0x7fce080aea40 con 0x7fce08074550 2024-04-25T00:55:49.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce0d55d640 1 -- 172.21.15.112:0/2973846491 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdf4003530 con 0x7fce08074550 2024-04-25T00:55:49.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce0dd5e640 1 -- 172.21.15.112:0/2973846491 >> v1:172.21.15.187:6789/0 conn(0x7fce080ade90 legacy=0x7fce08141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:49.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce0dd5e640 1 -- 172.21.15.112:0/2973846491 >> v1:172.21.15.112:6789/0 conn(0x7fce08074550 legacy=0x7fce0814aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.516+0000 7fce0dd5e640 1 -- 172.21.15.112:0/2973846491 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce08074930 con 0x7fce0814b2b0 2024-04-25T00:55:49.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0dd5e640 1 --2- 172.21.15.112:0/2973846491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce08140fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fce040097c0 tx=0x7fce0401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaa50f406aba3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0d55d640 1 -- 172.21.15.112:0/2973846491 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040ab680 con 0x7fce0814b2b0 2024-04-25T00:55:49.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0d55d640 1 -- 172.21.15.112:0/2973846491 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce040b0070 con 0x7fce0814b2b0 2024-04-25T00:55:49.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0d55d640 1 -- 172.21.15.112:0/2973846491 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040aba20 con 0x7fce0814b2b0 2024-04-25T00:55:49.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- 172.21.15.112:0/2973846491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 msgr2=0x7fce08140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 --2- 172.21.15.112:0/2973846491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce08140fd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fce040097c0 tx=0x7fce0401c460 comp rx=0 tx=0).stop 2024-04-25T00:55:49.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- 172.21.15.112:0/2973846491 shutdown_connections 2024-04-25T00:55:49.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 --2- 172.21.15.112:0/2973846491 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce08140fd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- 172.21.15.112:0/2973846491 >> 172.21.15.112:0/2973846491 conn(0x7fce080b91d0 msgr2=0x7fce080b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:49.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- 172.21.15.112:0/2973846491 shutdown_connections 2024-04-25T00:55:49.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- 172.21.15.112:0/2973846491 wait complete. 2024-04-25T00:55:49.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 Processor -- start 2024-04-25T00:55:49.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- start start 2024-04-25T00:55:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce08074550 0x7fce08149af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce080ade90 0x7fce081e3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce0812e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fce08155940 con 0x7fce0814b2b0 2024-04-25T00:55:49.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fce081557c0 con 0x7fce080ade90 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce107c3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fce08155ac0 con 0x7fce08074550 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce0812e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce0812e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51530/0 (socket says 172.21.15.112:51530) 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 -- 172.21.15.112:0/1554365880 learned_addr learned my addr 172.21.15.112:0/1554365880 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0e55f640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce08074550 0x7fce08149af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0dd5e640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce080ade90 0x7fce081e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 -- 172.21.15.112:0/1554365880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce08074550 msgr2=0x7fce08149af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce08074550 0x7fce08149af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 -- 172.21.15.112:0/1554365880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce080ade90 msgr2=0x7fce081e3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce080ade90 0x7fce081e3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 -- 172.21.15.112:0/1554365880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce080b1090 con 0x7fce0814b2b0 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.520+0000 7fce0ed60640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce0812e410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fce0000c7b0 tx=0x7fce0000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fcdfeffd640 1 -- 172.21.15.112:0/1554365880 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce00004b70 con 0x7fce0814b2b0 2024-04-25T00:55:49.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fcdfeffd640 1 -- 172.21.15.112:0/1554365880 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce00004d10 con 0x7fce0814b2b0 2024-04-25T00:55:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce080b2370 con 0x7fce0814b2b0 2024-04-25T00:55:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fcdfeffd640 1 -- 172.21.15.112:0/1554365880 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0000aa90 con 0x7fce0814b2b0 2024-04-25T00:55:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce080b0d60 con 0x7fce0814b2b0 2024-04-25T00:55:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce08144e20 con 0x7fce0814b2b0 2024-04-25T00:55:49.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fcdfeffd640 1 -- 172.21.15.112:0/1554365880 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fce0000ac30 con 0x7fce0814b2b0 2024-04-25T00:55:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fcdfeffd640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcdec0678c0 0x7fcdec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fcdfeffd640 1 -- 172.21.15.112:0/1554365880 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fce00085eb0 con 0x7fce0814b2b0 2024-04-25T00:55:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fce0e55f640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcdec0678c0 0x7fcdec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.524+0000 7fce0e55f640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcdec0678c0 0x7fcdec069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8005890 tx=0x7fcdf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.528+0000 7fcdfeffd640 1 -- 172.21.15.112:0/1554365880 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fce00020070 con 0x7fce0814b2b0 2024-04-25T00:55:49.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce0804f5f0 con 0x7fce0814b2b0 2024-04-25T00:55:49.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fcdfeffd640 1 -- 172.21.15.112:0/1554365880 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fce00056a80 con 0x7fce0814b2b0 2024-04-25T00:55:49.687 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:55:49.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcdec0678c0 msgr2=0x7fcdec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcdec0678c0 0x7fcdec069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8005890 tx=0x7fcdf8009040 comp rx=0 tx=0).stop 2024-04-25T00:55:49.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 msgr2=0x7fce0812e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce0812e410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fce0000c7b0 tx=0x7fce0000cc70 comp rx=0 tx=0).stop 2024-04-25T00:55:49.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 shutdown_connections 2024-04-25T00:55:49.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fce0814b2b0 0x7fce0812e410 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcdec0678c0 0x7fcdec069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fce080ade90 0x7fce081e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 --2- 172.21.15.112:0/1554365880 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fce08074550 0x7fce08149af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 >> 172.21.15.112:0/1554365880 conn(0x7fce080b91d0 msgr2=0x7fce0812ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 shutdown_connections 2024-04-25T00:55:49.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.684+0000 7fce107c3640 1 -- 172.21.15.112:0/1554365880 wait complete. 2024-04-25T00:55:49.705 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:55:49.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78cc653640 1 Processor -- start 2024-04-25T00:55:49.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78cc653640 1 -- start start 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78cc653640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f78c40b6800 0x7f78c41493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78cc653640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f78c4071100 0x7f78c40b4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78cc653640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c406d490 con 0x7f78c40b4a50 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78cc653640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f78c4149930 con 0x7f78c4071100 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78cc653640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f78c41489a0 con 0x7f78c40b6800 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c9bee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f78c4071100 0x7f78c40b4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c9bee640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f78c4071100 0x7f78c40b4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51546/0 (socket says 172.21.15.112:51546) 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c9bee640 1 -- 172.21.15.112:0/446142360 learned_addr learned my addr 172.21.15.112:0/446142360 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c9bee640 1 -- 172.21.15.112:0/446142360 >> v2:172.21.15.187:3300/0 conn(0x7f78c40b6800 msgr2=0x7f78c41493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:49.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78ca3ef640 1 --2- 172.21.15.112:0/446142360 >> v2:172.21.15.187:3300/0 conn(0x7f78c40b6800 0x7f78c41493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c9bee640 1 --2- 172.21.15.112:0/446142360 >> v2:172.21.15.187:3300/0 conn(0x7f78c40b6800 0x7f78c41493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c9bee640 1 -- 172.21.15.112:0/446142360 >> v1:172.21.15.187:6789/0 conn(0x7f78c40b4a50 legacy=0x7f78c41453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:49.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c9bee640 1 -- 172.21.15.112:0/446142360 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c40b6be0 con 0x7f78c4071100 2024-04-25T00:55:49.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c9bee640 1 --2- 172.21.15.112:0/446142360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c4071100 0x7f78c40b4510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78c40739a0 tx=0x7f78b401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a6205106917e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c93ed640 1 -- 172.21.15.112:0/446142360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40b2070 con 0x7f78c4071100 2024-04-25T00:55:49.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c93ed640 1 -- 172.21.15.112:0/446142360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b40ad640 con 0x7f78c4071100 2024-04-25T00:55:49.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.812+0000 7f78c93ed640 1 -- 172.21.15.112:0/446142360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40ada10 con 0x7f78c4071100 2024-04-25T00:55:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- 172.21.15.112:0/446142360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c4071100 msgr2=0x7f78c40b4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 --2- 172.21.15.112:0/446142360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c4071100 0x7f78c40b4510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f78c40739a0 tx=0x7f78b401c390 comp rx=0 tx=0).stop 2024-04-25T00:55:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- 172.21.15.112:0/446142360 shutdown_connections 2024-04-25T00:55:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 --2- 172.21.15.112:0/446142360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c4071100 0x7f78c40b4510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 --2- 172.21.15.112:0/446142360 >> v2:172.21.15.187:3300/0 conn(0x7f78c40b6800 0x7f78c41493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- 172.21.15.112:0/446142360 >> 172.21.15.112:0/446142360 conn(0x7f78c40baaf0 msgr2=0x7f78c40baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- 172.21.15.112:0/446142360 shutdown_connections 2024-04-25T00:55:49.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- 172.21.15.112:0/446142360 wait complete. 2024-04-25T00:55:49.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 Processor -- start 2024-04-25T00:55:49.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- start start 2024-04-25T00:55:49.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78c4071100 0x7f78c4134850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c40b4a50 0x7f78c4134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78c40b6800 0x7f78c41ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40b5ad0 con 0x7f78c40b4a50 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f78c40b5950 con 0x7f78c4071100 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f78c40b5da0 con 0x7f78c40b6800 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c40b4a50 0x7f78c4134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c40b4a50 0x7f78c4134d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51548/0 (socket says 172.21.15.112:51548) 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 -- 172.21.15.112:0/3972379425 learned_addr learned my addr 172.21.15.112:0/3972379425 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 -- 172.21.15.112:0/3972379425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78c40b6800 msgr2=0x7f78c41ddcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78c40b6800 0x7f78c41ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 -- 172.21.15.112:0/3972379425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78c4071100 msgr2=0x7f78c4134850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78c4071100 0x7f78c4134850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 -- 172.21.15.112:0/3972379425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c413d300 con 0x7f78c40b4a50 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78c9bee640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c40b4a50 0x7f78c4134d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78b4019fc0 tx=0x7f78b4019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78baffd640 1 -- 172.21.15.112:0/3972379425 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40bd070 con 0x7f78c40b4a50 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78baffd640 1 -- 172.21.15.112:0/3972379425 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78b40b6770 con 0x7f78c40b4a50 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c4144910 con 0x7f78c40b4a50 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78baffd640 1 -- 172.21.15.112:0/3972379425 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b40b2070 con 0x7f78c40b4a50 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c41486a0 con 0x7f78c40b4a50 2024-04-25T00:55:49.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.816+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c404f580 con 0x7f78c40b4a50 2024-04-25T00:55:49.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.820+0000 7f78baffd640 1 -- 172.21.15.112:0/3972379425 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f78b40c0070 con 0x7f78c40b4a50 2024-04-25T00:55:49.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.820+0000 7f78baffd640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f78a80678c0 0x7f78a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:49.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.820+0000 7f78baffd640 1 -- 172.21.15.112:0/3972379425 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f78b411fa30 con 0x7f78c40b4a50 2024-04-25T00:55:49.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.820+0000 7f78ca3ef640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f78a80678c0 0x7f78a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:49.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.820+0000 7f78ca3ef640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f78a80678c0 0x7f78a8069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f78c000a8b0 tx=0x7f78c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:49.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:49.824+0000 7f78baffd640 1 -- 172.21.15.112:0/3972379425 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78b40160d0 con 0x7f78c40b4a50 2024-04-25T00:55:50.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.004+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78c40b6f50 con 0x7f78c40b4a50 2024-04-25T00:55:50.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.004+0000 7f78baffd640 1 -- 172.21.15.112:0/3972379425 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f78b40f0560 con 0x7f78c40b4a50 2024-04-25T00:55:50.009 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:50.009 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:55:38.966915+0000","modified":"2024-04-25T00:55:42.002877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:55:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T00:55:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f78a80678c0 msgr2=0x7f78a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f78a80678c0 0x7f78a8069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f78c000a8b0 tx=0x7f78c0008040 comp rx=0 tx=0).stop 2024-04-25T00:55:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c40b4a50 msgr2=0x7f78c4134d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c40b4a50 0x7f78c4134d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f78b4019fc0 tx=0x7f78b4019130 comp rx=0 tx=0).stop 2024-04-25T00:55:50.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 shutdown_connections 2024-04-25T00:55:50.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f78c40b6800 0x7f78c41ddcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78c40b4a50 0x7f78c4134d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f78a80678c0 0x7f78a8069d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 --2- 172.21.15.112:0/3972379425 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f78c4071100 0x7f78c4134850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 >> 172.21.15.112:0/3972379425 conn(0x7f78c40baaf0 msgr2=0x7f78c40b7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:50.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 shutdown_connections 2024-04-25T00:55:50.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.008+0000 7f78cc653640 1 -- 172.21.15.112:0/3972379425 wait complete. 2024-04-25T00:55:50.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:50.138 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2b0124d640 1 Processor -- start 2024-04-25T00:55:50.139 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2b0124d640 1 -- start start 2024-04-25T00:55:50.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2b0124d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2afc072530 0x7f2afc0b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2b0124d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f2afc1459c0 0x7f2afc145e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2b0124d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2afc0b4bf0 con 0x7f2afc145370 2024-04-25T00:55:50.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2b0124d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f2afc146350 con 0x7f2afc072530 2024-04-25T00:55:50.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2b0124d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2afc0b5270 con 0x7f2afc1459c0 2024-04-25T00:55:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2afc072530 0x7f2afc0b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2afc072530 0x7f2afc0b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51562/0 (socket says 172.21.15.112:51562) 2024-04-25T00:55:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afad9d640 1 -- 172.21.15.112:0/1968903067 learned_addr learned my addr 172.21.15.112:0/1968903067 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afb59e640 1 --2- 172.21.15.112:0/1968903067 >> v2:172.21.15.187:3300/0 conn(0x7f2afc1459c0 0x7f2afc145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afad9d640 1 -- 172.21.15.112:0/1968903067 >> v2:172.21.15.187:3300/0 conn(0x7f2afc1459c0 msgr2=0x7f2afc145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afad9d640 1 --2- 172.21.15.112:0/1968903067 >> v2:172.21.15.187:3300/0 conn(0x7f2afc1459c0 0x7f2afc145e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afad9d640 1 -- 172.21.15.112:0/1968903067 >> v1:172.21.15.112:6789/0 conn(0x7f2afc145370 legacy=0x7f2afc0b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.142 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afad9d640 1 -- 172.21.15.112:0/1968903067 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc0b4e90 con 0x7f2afc072530 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.136+0000 7f2afad9d640 1 --2- 172.21.15.112:0/1968903067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc072530 0x7f2afc0b3a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0732c0 tx=0x7f2aec01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5e079925128225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2afa59c640 1 -- 172.21.15.112:0/1968903067 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0b0070 con 0x7f2afc072530 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2afa59c640 1 -- 172.21.15.112:0/1968903067 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aec0ab600 con 0x7f2afc072530 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2afa59c640 1 -- 172.21.15.112:0/1968903067 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0ab9d0 con 0x7f2afc072530 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- 172.21.15.112:0/1968903067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc072530 msgr2=0x7f2afc0b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 --2- 172.21.15.112:0/1968903067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc072530 0x7f2afc0b3a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0732c0 tx=0x7f2aec01c400 comp rx=0 tx=0).stop 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- 172.21.15.112:0/1968903067 shutdown_connections 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 --2- 172.21.15.112:0/1968903067 >> v2:172.21.15.187:3300/0 conn(0x7f2afc1459c0 0x7f2afc145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 --2- 172.21.15.112:0/1968903067 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc072530 0x7f2afc0b3a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- 172.21.15.112:0/1968903067 >> 172.21.15.112:0/1968903067 conn(0x7f2afc06b210 msgr2=0x7f2afc06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- 172.21.15.112:0/1968903067 shutdown_connections 2024-04-25T00:55:50.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- 172.21.15.112:0/1968903067 wait complete. 2024-04-25T00:55:50.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 Processor -- start 2024-04-25T00:55:50.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- start start 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2afc072530 0x7f2afc137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc145370 0x7f2afc138190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2afc1459c0 0x7f2afc1e1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc152840 con 0x7f2afc145370 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1526c0 con 0x7f2afc1459c0 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc1529c0 con 0x7f2afc072530 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc145370 0x7f2afc138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc145370 0x7f2afc138190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51570/0 (socket says 172.21.15.112:51570) 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 -- 172.21.15.112:0/2146765430 learned_addr learned my addr 172.21.15.112:0/2146765430 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:50.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2afad9d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2afc072530 0x7f2afc137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2afb59e640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2afc1459c0 0x7f2afc1e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 -- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2afc072530 msgr2=0x7f2afc137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2afc072530 0x7f2afc137c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 -- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2afc1459c0 msgr2=0x7f2afc1e1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2afc1459c0 0x7f2afc1e1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 -- 172.21.15.112:0/2146765430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc13df10 con 0x7f2afc145370 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2afb59e640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2afc1459c0 0x7f2afc1e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af3fff640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc145370 0x7f2afc138190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c3c0 tx=0x7f2ae400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af37fe640 1 -- 172.21.15.112:0/2146765430 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4017070 con 0x7f2afc145370 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af37fe640 1 -- 172.21.15.112:0/2146765430 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae40052d0 con 0x7f2afc145370 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2af37fe640 1 -- 172.21.15.112:0/2146765430 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4003840 con 0x7f2afc145370 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc0a96b0 con 0x7f2afc145370 2024-04-25T00:55:50.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.140+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc13e3c0 con 0x7f2afc145370 2024-04-25T00:55:50.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.144+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc04f580 con 0x7f2afc145370 2024-04-25T00:55:50.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.144+0000 7f2af37fe640 1 -- 172.21.15.112:0/2146765430 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ae401a070 con 0x7f2afc145370 2024-04-25T00:55:50.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.144+0000 7f2af37fe640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2adc0678f0 0x7f2adc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.144+0000 7f2af37fe640 1 -- 172.21.15.112:0/2146765430 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ae4084fa0 con 0x7f2afc145370 2024-04-25T00:55:50.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.144+0000 7f2afad9d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2adc0678f0 0x7f2adc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.144+0000 7f2afad9d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2adc0678f0 0x7f2adc069db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0ad000 tx=0x7f2aec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.153 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.148+0000 7f2af37fe640 1 -- 172.21.15.112:0/2146765430 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2ae4005480 con 0x7f2afc145370 2024-04-25T00:55:50.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.296+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2afc144040 con 0x7f2afc145370 2024-04-25T00:55:50.304 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2af37fe640 1 -- 172.21.15.112:0/2146765430 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f2ae4055af0 con 0x7f2afc145370 2024-04-25T00:55:50.304 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:50.304 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2adc0678f0 msgr2=0x7f2adc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2adc0678f0 0x7f2adc069db0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0ad000 tx=0x7f2aec008040 comp rx=0 tx=0).stop 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc145370 msgr2=0x7f2afc138190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc145370 0x7f2afc138190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c3c0 tx=0x7f2ae400c880 comp rx=0 tx=0).stop 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 shutdown_connections 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f2adc0678f0 0x7f2adc069db0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2afc1459c0 0x7f2afc1e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2afc145370 0x7f2afc138190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.306 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 --2- 172.21.15.112:0/2146765430 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2afc072530 0x7f2afc137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 >> 172.21.15.112:0/2146765430 conn(0x7f2afc06b210 msgr2=0x7f2afc0ba190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:50.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 shutdown_connections 2024-04-25T00:55:50.307 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.300+0000 7f2b0124d640 1 -- 172.21.15.112:0/2146765430 wait complete. 2024-04-25T00:55:50.322 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:55:50.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.424+0000 7f0e5cfad640 1 Processor -- start 2024-04-25T00:55:50.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.424+0000 7f0e5cfad640 1 -- start start 2024-04-25T00:55:50.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.424+0000 7f0e5cfad640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0e580ae3b0 0x7f0e580b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.424+0000 7f0e5cfad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580b2780 0x7f0e5814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.424+0000 7f0e5cfad640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e580b52f0 con 0x7f0e580adfd0 2024-04-25T00:55:50.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.424+0000 7f0e5cfad640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0e580af0b0 con 0x7f0e580ae3b0 2024-04-25T00:55:50.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.424+0000 7f0e5cfad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e580b0800 con 0x7f0e580b2780 2024-04-25T00:55:50.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e56d9d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580b2780 0x7f0e5814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e55d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0e580ae3b0 0x7f0e580b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e55d9b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0e580ae3b0 0x7f0e580b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:41326/0 (socket says 172.21.15.112:41326) 2024-04-25T00:55:50.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e55d9b640 1 -- 172.21.15.112:0/2830156850 learned_addr learned my addr 172.21.15.112:0/2830156850 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:50.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e55d9b640 1 -- 172.21.15.112:0/2830156850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580b2780 msgr2=0x7f0e5814e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:50.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e55d9b640 1 --2- 172.21.15.112:0/2830156850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580b2780 0x7f0e5814e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e55d9b640 1 -- 172.21.15.112:0/2830156850 >> v1:172.21.15.187:6789/0 conn(0x7f0e580adfd0 legacy=0x7f0e580ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e55d9b640 1 -- 172.21.15.112:0/2830156850 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e58143f50 con 0x7f0e580ae3b0 2024-04-25T00:55:50.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e55d9b640 1 --2- 172.21.15.112:0/2830156850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580ae3b0 0x7f0e580b2240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e400098a0 tx=0x7f0e4001c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832693bd26c2bdb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5559a640 1 -- 172.21.15.112:0/2830156850 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e400b0070 con 0x7f0e580ae3b0 2024-04-25T00:55:50.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5559a640 1 -- 172.21.15.112:0/2830156850 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e400ab600 con 0x7f0e580ae3b0 2024-04-25T00:55:50.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5559a640 1 -- 172.21.15.112:0/2830156850 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e400ab9d0 con 0x7f0e580ae3b0 2024-04-25T00:55:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2830156850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580ae3b0 msgr2=0x7f0e580b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2830156850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580ae3b0 0x7f0e580b2240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e400098a0 tx=0x7f0e4001c4b0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2830156850 shutdown_connections 2024-04-25T00:55:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2830156850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580b2780 0x7f0e5814e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2830156850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580ae3b0 0x7f0e580b2240 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.428+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2830156850 >> 172.21.15.112:0/2830156850 conn(0x7f0e580b5980 msgr2=0x7f0e580b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:50.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2830156850 shutdown_connections 2024-04-25T00:55:50.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2830156850 wait complete. 2024-04-25T00:55:50.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 Processor -- start 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- start start 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580adfd0 0x7f0e58144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e580ae3b0 0x7f0e5813cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580b2780 0x7f0e581df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0e581508a0 con 0x7f0e580ae3b0 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58150720 con 0x7f0e580b2780 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0e58150a20 con 0x7f0e580adfd0 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e580ae3b0 0x7f0e5813cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e580ae3b0 0x7f0e5813cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51576/0 (socket says 172.21.15.112:51576) 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 -- 172.21.15.112:0/2518247756 learned_addr learned my addr 172.21.15.112:0/2518247756 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5659c640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580adfd0 0x7f0e58144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e56d9d640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580b2780 0x7f0e581df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 -- 172.21.15.112:0/2518247756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580adfd0 msgr2=0x7f0e58144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580adfd0 0x7f0e58144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 -- 172.21.15.112:0/2518247756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580b2780 msgr2=0x7f0e581df3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580b2780 0x7f0e581df3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 -- 172.21.15.112:0/2518247756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e58141f30 con 0x7f0e580ae3b0 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e55d9b640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e580ae3b0 0x7f0e5813cc40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0e4001a0e0 tx=0x7f0e40019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e4effd640 1 -- 172.21.15.112:0/2518247756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e400b0040 con 0x7f0e580ae3b0 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e4effd640 1 -- 172.21.15.112:0/2518247756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e400b43f0 con 0x7f0e580ae3b0 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e58142210 con 0x7f0e580ae3b0 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e4effd640 1 -- 172.21.15.112:0/2518247756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e400b47c0 con 0x7f0e580ae3b0 2024-04-25T00:55:50.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e58142a10 con 0x7f0e580ae3b0 2024-04-25T00:55:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e5804f580 con 0x7f0e580ae3b0 2024-04-25T00:55:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.432+0000 7f0e4effd640 1 -- 172.21.15.112:0/2518247756 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0e400b4960 con 0x7f0e580ae3b0 2024-04-25T00:55:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.436+0000 7f0e4effd640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0e380678f0 0x7f0e38069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.436+0000 7f0e5659c640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0e380678f0 0x7f0e38069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.436+0000 7f0e4effd640 1 -- 172.21.15.112:0/2518247756 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0e4011f620 con 0x7f0e580ae3b0 2024-04-25T00:55:50.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.436+0000 7f0e5659c640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0e380678f0 0x7f0e38069db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e48000f70 tx=0x7f0e48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.442 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.440+0000 7f0e4effd640 1 -- 172.21.15.112:0/2518247756 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0e40016130 con 0x7f0e580ae3b0 2024-04-25T00:55:50.627 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e580b8fc0 con 0x7f0e580ae3b0 2024-04-25T00:55:50.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e4effd640 1 -- 172.21.15.112:0/2518247756 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f0e400f01f0 con 0x7f0e580ae3b0 2024-04-25T00:55:50.628 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:50.628 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:55:38.966915+0000","modified":"2024-04-25T00:55:42.002877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:55:50.628 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0e380678f0 msgr2=0x7f0e38069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0e380678f0 0x7f0e38069db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e48000f70 tx=0x7f0e48009040 comp rx=0 tx=0).stop 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e580ae3b0 msgr2=0x7f0e5813cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e580ae3b0 0x7f0e5813cc40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0e4001a0e0 tx=0x7f0e40019250 comp rx=0 tx=0).stop 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 shutdown_connections 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0e380678f0 0x7f0e38069db0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0e580b2780 0x7f0e581df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e580ae3b0 0x7f0e5813cc40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 --2- 172.21.15.112:0/2518247756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0e580adfd0 0x7f0e58144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 >> 172.21.15.112:0/2518247756 conn(0x7f0e580b5980 msgr2=0x7f0e580b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 shutdown_connections 2024-04-25T00:55:50.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.624+0000 7f0e5cfad640 1 -- 172.21.15.112:0/2518247756 wait complete. 2024-04-25T00:55:50.847 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:55:38.966915+0000', 'modified': '2024-04-25T00:55:42.002877+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.187:6833/2006003648', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.187:6832', 'nonce': 2006003648}, {'type': 'v1', 'addr': '172.21.15.187:6833', 'nonce': 2006003648}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4270, 'qdb_cluster': [4270]} 2024-04-25T00:55:50.847 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:55:50.847 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:55:50.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f53981640 1 Processor -- start 2024-04-25T00:55:50.971 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f53981640 1 -- start start 2024-04-25T00:55:50.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f53981640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f4c0b23f0 0x7f0f4c0add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f53981640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0f4c145200 0x7f0f4c0b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f53981640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f4c0aeec0 con 0x7f0f4c072760 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f53981640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0f4c0af780 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f53981640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0f4c0b27d0 con 0x7f0f4c145200 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f5171d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0f4c0b23f0 0x7f0f4c0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f51f1e640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0f4c072760 0x7f0f4c06c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:57358/0 (socket says 172.21.15.112:57358) 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f51f1e640 1 -- 172.21.15.112:0/4183024234 learned_addr learned my addr 172.21.15.112:0/4183024234 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/4183024234 >> v2:172.21.15.187:3300/0 conn(0x7f0f4c145200 0x7f0f4c0b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f5171d640 1 -- 172.21.15.112:0/4183024234 >> v2:172.21.15.187:3300/0 conn(0x7f0f4c145200 msgr2=0x7f0f4c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f5171d640 1 --2- 172.21.15.112:0/4183024234 >> v2:172.21.15.187:3300/0 conn(0x7f0f4c145200 0x7f0f4c0b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f5171d640 1 -- 172.21.15.112:0/4183024234 >> v1:172.21.15.112:6789/0 conn(0x7f0f4c072760 legacy=0x7f0f4c06c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f5171d640 1 -- 172.21.15.112:0/4183024234 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c0afa20 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f43fff640 1 -- 172.21.15.112:0/4183024234 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902073814 0 0) 0x7f0f4c0aeec0 con 0x7f0f4c072760 2024-04-25T00:55:50.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.968+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/4183024234 >> v2:172.21.15.187:3300/0 conn(0x7f0f4c145200 0x7f0f4c0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f5171d640 1 --2- 172.21.15.112:0/4183024234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c0add70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0aece0 tx=0x7f0f4801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b32a35286504ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f43fff640 1 -- 172.21.15.112:0/4183024234 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480b1070 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f43fff640 1 -- 172.21.15.112:0/4183024234 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f480ac5f0 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f43fff640 1 -- 172.21.15.112:0/4183024234 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480ac9c0 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- 172.21.15.112:0/4183024234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 msgr2=0x7f0f4c0add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 --2- 172.21.15.112:0/4183024234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c0add70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0aece0 tx=0x7f0f4801c480 comp rx=0 tx=0).stop 2024-04-25T00:55:50.975 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- 172.21.15.112:0/4183024234 shutdown_connections 2024-04-25T00:55:50.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 --2- 172.21.15.112:0/4183024234 >> v2:172.21.15.187:3300/0 conn(0x7f0f4c145200 0x7f0f4c0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 --2- 172.21.15.112:0/4183024234 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c0add70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- 172.21.15.112:0/4183024234 >> 172.21.15.112:0/4183024234 conn(0x7f0f4c0b5980 msgr2=0x7f0f4c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:50.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- 172.21.15.112:0/4183024234 shutdown_connections 2024-04-25T00:55:50.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- 172.21.15.112:0/4183024234 wait complete. 2024-04-25T00:55:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 Processor -- start 2024-04-25T00:55:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- start start 2024-04-25T00:55:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f4c072760 0x7f0f4c1427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c140500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f4c145200 0x7f0f4c140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c072f50 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c0b5360 con 0x7f0f4c072760 2024-04-25T00:55:50.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4c152d70 con 0x7f0f4c145200 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c140500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c140500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51596/0 (socket says 172.21.15.112:51596) 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 -- 172.21.15.112:0/3848215137 learned_addr learned my addr 172.21.15.112:0/3848215137 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f5171d640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f4c072760 0x7f0f4c1427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f51f1e640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f4c145200 0x7f0f4c140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 -- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f4c145200 msgr2=0x7f0f4c140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f4c145200 0x7f0f4c140a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 -- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f4c072760 msgr2=0x7f0f4c1427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f4c072760 0x7f0f4c1427b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 -- 172.21.15.112:0/3848215137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4c141d50 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f5171d640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f4c072760 0x7f0f4c1427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c140500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c820 tx=0x7f0f3c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f41ffb640 1 -- 172.21.15.112:0/3848215137 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c015070 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f41ffb640 1 -- 172.21.15.112:0/3848215137 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c007a30 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- 172.21.15.112:0/3848215137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4c13b950 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f41ffb640 1 -- 172.21.15.112:0/3848215137 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c007e40 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.972+0000 7f0f53981640 1 -- 172.21.15.112:0/3848215137 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f4c13a7e0 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f41ffb640 1 -- 172.21.15.112:0/3848215137 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0f3c018070 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f41ffb640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0f300678c0 0x7f0f30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f41ffb640 1 -- 172.21.15.112:0/3848215137 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0f3c085b70 con 0x7f0f4c0b23f0 2024-04-25T00:55:50.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f5171d640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0f300678c0 0x7f0f30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f56afa000 1 Processor -- start 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f5171d640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0f300678c0 0x7f0f30069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0f480ae000 tx=0x7f0f48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f56afa000 1 -- start start 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b56484bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56484c140 0x55b564876310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 0x55b564876c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f56afa000 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b5646945e0 con 0x55b56484c140 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f56afa000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b56461bc50 con 0x55b564877d90 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f56afa000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55b5646905a0 con 0x55b564876850 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56484c140 0x55b564876310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56484c140 0x55b564876310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51606/0 (socket says 172.21.15.112:51606) 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 -- 172.21.15.112:0/1765570724 learned_addr learned my addr 172.21.15.112:0/1765570724 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 -- 172.21.15.112:0/1765570724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 msgr2=0x55b564876c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:50.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f51f1e640 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 0x55b564876c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f5171d640 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b56484bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 0x55b564876c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 -- 172.21.15.112:0/1765570724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 msgr2=0x55b56484bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b56484bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 -- 172.21.15.112:0/1765570724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b564619d10 con 0x55b56484c140 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56484c140 0x55b564876310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c006ee0 tx=0x7f0f3c0560e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4a9b499b088289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f36ffd640 1 -- 172.21.15.112:0/1765570724 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c082070 con 0x55b56484c140 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f36ffd640 1 -- 172.21.15.112:0/1765570724 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c0542a0 con 0x55b56484c140 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.976+0000 7f0f36ffd640 1 -- 172.21.15.112:0/1765570724 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c01f040 con 0x55b56484c140 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- 172.21.15.112:0/1765570724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56484c140 msgr2=0x55b564876310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56484c140 0x55b564876310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c006ee0 tx=0x7f0f3c0560e0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- 172.21.15.112:0/1765570724 shutdown_connections 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 0x55b564876c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b56484c140 0x55b564876310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 --2- 172.21.15.112:0/1765570724 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b56484bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- 172.21.15.112:0/1765570724 >> 172.21.15.112:0/1765570724 conn(0x55b5648720c0 msgr2=0x55b5648744b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- 172.21.15.112:0/1765570724 shutdown_connections 2024-04-25T00:55:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- 172.21.15.112:0/1765570724 wait complete. 2024-04-25T00:55:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- start start 2024-04-25T00:55:50.986 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 Processor -- start 2024-04-25T00:55:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b56484bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b56484ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b564847cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b564698c10 con 0x55b564876850 2024-04-25T00:55:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b56464e520 con 0x55b564877d90 2024-04-25T00:55:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55b564661340 con 0x55b56484c140 2024-04-25T00:55:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 10 client.0 _ll_register_callbacks cb 0x55b5648315c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:55:50.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b56484ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b56484ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51620/0 (socket says 172.21.15.112:51620) 2024-04-25T00:55:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 -- 172.21.15.112:0/3184679140 learned_addr learned my addr 172.21.15.112:0/3184679140 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f5171d640 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b56484bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 -- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 msgr2=0x55b56484bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:50.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b56484bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 -- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 msgr2=0x55b564847cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f51f1e640 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b564847cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b564847cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 -- 172.21.15.112:0/3184679140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56462c7b0 con 0x55b564876850 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b56484ad20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007410 tx=0x7f0f3c006e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7680b9ec205b4404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f2bfff640 1 -- 172.21.15.112:0/3184679140 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c082070 con 0x55b564876850 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f2bfff640 1 -- 172.21.15.112:0/3184679140 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0f3c051cf0 con 0x55b564876850 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- 172.21.15.112:0/3184679140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f3c0542a0 con 0x55b564876850 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f2bfff640 1 -- 172.21.15.112:0/3184679140 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c01f040 con 0x55b564876850 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 10 client.4566 fetch_fsmap learned FSMap version 8 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- 172.21.15.112:0/3184679140 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55b56461bea0 con 0x55b564876850 2024-04-25T00:55:50.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f2bfff640 1 -- 172.21.15.112:0/3184679140 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f3c0542a0 con 0x55b564876850 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f2bfff640 1 -- 172.21.15.112:0/3184679140 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 8) ==== 2284+0+0 (secure 0 0 0) 0x7f0f3c051ef0 con 0x55b564876850 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 10 client.4566 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 10 client.4566 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 20 client.4566 populate_metadata read hostname 'smithi112' 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] conn(0x55b564848700 0x55b56488d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 4 client.4566 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f56afa000 1 -- 172.21.15.112:0/3184679140 --> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b56462fab0 con 0x55b564848700 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f5171d640 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] conn(0x55b564848700 0x55b56488d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f5171d640 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] conn(0x55b564848700 0x55b56488d560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1343c0e3bd4ac352 server_cookie=e080bb3b45090879 in_seq=0 out_seq=0 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.980+0000 7f0f2bfff640 10 client.4566 ms_handle_connect on v2:172.21.15.187:6832/2006003648 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f2bfff640 1 -- 172.21.15.112:0/3184679140 <== mds.0 v2:172.21.15.187:6832/2006003648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b56462fab0 con 0x55b564848700 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f2bfff640 10 client.4566 handle_command_reply: tid=0 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 client.4566 shutdown 2024-04-25T00:55:50.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 -- 172.21.15.112:0/3184679140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 msgr2=0x55b56484ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b56484ad20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007410 tx=0x7f0f3c006e70 comp rx=0 tx=0).stop 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 -- 172.21.15.112:0/3184679140 shutdown_connections 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] conn(0x55b564848700 0x55b56488d560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b564847cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b56484ad20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 --2- 172.21.15.112:0/3184679140 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b56484bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f2bfff640 0 client.4566 ms_handle_reset on v2:172.21.15.187:6832/2006003648 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 -- 172.21.15.112:0/3184679140 >> 172.21.15.112:0/3184679140 conn(0x55b5648720c0 msgr2=0x55b5648746d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 -- 172.21.15.112:0/3184679140 shutdown_connections 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 1 -- 172.21.15.112:0/3184679140 wait complete. 2024-04-25T00:55:50.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:50.984+0000 7f0f56afa000 20 client.4566 trim_cache size 0 max 16384 2024-04-25T00:55:51.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.000+0000 7f0f56afa000 1 Processor -- start 2024-04-25T00:55:51.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.000+0000 7f0f56afa000 1 -- start start 2024-04-25T00:55:51.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.000+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b56484c140 0x55b5648774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.000+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 0x55b564890590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.000+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564877d90 0x55b564890ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.000+0000 7f0f56afa000 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c1ea730 con 0x55b564877d90 2024-04-25T00:55:51.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.000+0000 7f0f56afa000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4c1e9890 con 0x55b56484c140 2024-04-25T00:55:51.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.000+0000 7f0f56afa000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55b5646938f0 con 0x55b564876850 2024-04-25T00:55:51.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564877d90 0x55b564890ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564877d90 0x55b564890ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51634/0 (socket says 172.21.15.112:51634) 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 -- 172.21.15.112:0/1337149503 learned_addr learned my addr 172.21.15.112:0/1337149503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 -- 172.21.15.112:0/1337149503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 msgr2=0x55b564890590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 0x55b564890590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 0x55b564890590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f5171d640 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b56484c140 0x55b5648774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 -- 172.21.15.112:0/1337149503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b56484c140 msgr2=0x55b5648774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b56484c140 0x55b5648774e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 -- 172.21.15.112:0/1337149503 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56461ce80 con 0x55b564877d90 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564877d90 0x55b564890ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0f44001ba0 tx=0x7f0f44002060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d975e0902fa102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f367fc640 1 -- 172.21.15.112:0/1337149503 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44019070 con 0x55b564877d90 2024-04-25T00:55:51.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f367fc640 1 -- 172.21.15.112:0/1337149503 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4400f8d0 con 0x55b564877d90 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f367fc640 1 -- 172.21.15.112:0/1337149503 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400fcc0 con 0x55b564877d90 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- 172.21.15.112:0/1337149503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564877d90 msgr2=0x55b564890ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564877d90 0x55b564890ad0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0f44001ba0 tx=0x7f0f44002060 comp rx=0 tx=0).stop 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- 172.21.15.112:0/1337149503 shutdown_connections 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564877d90 0x55b564890ad0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b564876850 0x55b564890590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 --2- 172.21.15.112:0/1337149503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b56484c140 0x55b5648774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- 172.21.15.112:0/1337149503 >> 172.21.15.112:0/1337149503 conn(0x55b5648720c0 msgr2=0x55b564888a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- 172.21.15.112:0/1337149503 shutdown_connections 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- 172.21.15.112:0/1337149503 wait complete. 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- start start 2024-04-25T00:55:51.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 Processor -- start 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b564871700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b564890590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b564890ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55b564698c10 con 0x55b564876850 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55b564661340 con 0x55b564877d90 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55b56464e520 con 0x55b56484c140 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 10 client.0 _ll_register_callbacks cb 0x7f0f3c080070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b564890590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b564890590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51642/0 (socket says 172.21.15.112:51642) 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 -- 172.21.15.112:0/985199021 learned_addr learned my addr 172.21.15.112:0/985199021 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f5171d640 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b564871700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 -- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 msgr2=0x55b564871700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b564871700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f51f1e640 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b564890ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 -- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 msgr2=0x55b564890ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b564890ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 -- 172.21.15.112:0/985199021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b564661760 con 0x55b564876850 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f5171d640 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b564871700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f50f1c640 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b564890590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c056850 tx=0x7f0f3c055df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6fdc8fca364228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f2bfff640 1 -- 172.21.15.112:0/985199021 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c082040 con 0x55b564876850 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f2bfff640 1 -- 172.21.15.112:0/985199021 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0f3c051f00 con 0x55b564876850 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- 172.21.15.112:0/985199021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f4400f8d0 con 0x55b564876850 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f2bfff640 1 -- 172.21.15.112:0/985199021 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c00ce10 con 0x55b564876850 2024-04-25T00:55:51.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 10 client.4572 fetch_fsmap learned FSMap version 8 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- 172.21.15.112:0/985199021 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55b5646ed9e0 con 0x55b564876850 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f2bfff640 1 -- 172.21.15.112:0/985199021 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4400f8d0 con 0x55b564876850 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f2bfff640 1 -- 172.21.15.112:0/985199021 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 8) ==== 2284+0+0 (secure 0 0 0) 0x7f0f3c07d9f0 con 0x55b564876850 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 10 client.4572 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 10 client.4572 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 20 client.4572 populate_metadata read hostname 'smithi112' 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] conn(0x55b564843330 0x55b564843710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 4 client.4572 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f56afa000 1 -- 172.21.15.112:0/985199021 --> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] -- command(tid 0: {"prefix": "status"}) -- 0x55b5646a1b20 con 0x55b564843330 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.004+0000 7f0f5171d640 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] conn(0x55b564843330 0x55b564843710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f5171d640 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] conn(0x55b564843330 0x55b564843710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bedfd3b9b61a81e server_cookie=ffac434b8e3f305d in_seq=0 out_seq=0 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f2bfff640 10 client.4572 ms_handle_connect on v2:172.21.15.187:6832/2006003648 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f2bfff640 1 -- 172.21.15.112:0/985199021 <== mds.0 v2:172.21.15.187:6832/2006003648 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55b5646a1b20 con 0x55b564843330 2024-04-25T00:55:51.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f2bfff640 10 client.4572 handle_command_reply: tid=0 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 client.4572 shutdown 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 -- 172.21.15.112:0/985199021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 msgr2=0x55b564890590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b564890590 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c056850 tx=0x7f0f3c055df0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 -- 172.21.15.112:0/985199021 shutdown_connections 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:6832/2006003648,v1:172.21.15.187:6833/2006003648] conn(0x55b564843330 0x55b564843710 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55b564877d90 0x55b564890ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f2bfff640 0 client.4572 ms_handle_reset on v2:172.21.15.187:6832/2006003648 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55b564876850 0x55b564890590 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 --2- 172.21.15.112:0/985199021 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55b56484c140 0x55b564871700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 -- 172.21.15.112:0/985199021 >> 172.21.15.112:0/985199021 conn(0x55b5648720c0 msgr2=0x55b564870b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 -- 172.21.15.112:0/985199021 shutdown_connections 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 1 -- 172.21.15.112:0/985199021 wait complete. 2024-04-25T00:55:51.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f56afa000 20 client.4572 trim_cache size 0 max 16384 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "86f864a3-be50-4513-98ca-86783428a0f8", 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4270, 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 11.376686526, 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 8, 2024-04-25T00:55:51.020 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 20, 2024-04-25T00:55:51.021 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T00:55:51.021 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 17.966929698000001 2024-04-25T00:55:51.021 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:55:51.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 -- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0f300678c0 msgr2=0x7f0f30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0f300678c0 0x7f0f30069d80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0f480ae000 tx=0x7f0f48008040 comp rx=0 tx=0).stop 2024-04-25T00:55:51.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 -- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 msgr2=0x7f0f4c140500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c140500 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c00c820 tx=0x7f0f3c00cce0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 -- 172.21.15.112:0/3848215137 shutdown_connections 2024-04-25T00:55:51.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0f4c145200 0x7f0f4c140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0b23f0 0x7f0f4c140500 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0f300678c0 0x7f0f30069d80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 --2- 172.21.15.112:0/3848215137 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0f4c072760 0x7f0f4c1427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 -- 172.21.15.112:0/3848215137 >> 172.21.15.112:0/3848215137 conn(0x7f0f4c0b5980 msgr2=0x7f0f4c0b92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:51.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 -- 172.21.15.112:0/3848215137 shutdown_connections 2024-04-25T00:55:51.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.008+0000 7f0f53981640 1 -- 172.21.15.112:0/3848215137 wait complete. 2024-04-25T00:55:51.027 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:55:51.038 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:55:51.038 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:55:51.038 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:55:51.038 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T00:55:51.062 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T00:55:51.062 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-04-25T00:55:51.062 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T00:55:51.062 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:55:51.062 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:55:51.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:51.063 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T00:55:51.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:51.068 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T00:55:51.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:55:51.126 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2024-04-25T00:55:51.126 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T00:55:51.127 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:55:51.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c328c640 1 Processor -- start 2024-04-25T00:55:51.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c328c640 1 -- start start 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c328c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc06c630 0x7f36bc0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c328c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f36bc0add70 0x7f36bc0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c328c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36bc0af760 con 0x7f36bc06ca10 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c328c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f36bc148860 con 0x7f36bc0add70 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c328c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36bc0b52f0 con 0x7f36bc06c630 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c1829640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f36bc0add70 0x7f36bc0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c1028640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc06c630 0x7f36bc0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c1829640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f36bc0add70 0x7f36bc0b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51652/0 (socket says 172.21.15.112:51652) 2024-04-25T00:55:51.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c0827640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f36bc06ca10 0x7f36bc1477c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:52830/0 (socket says 172.21.15.112:52830) 2024-04-25T00:55:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c1829640 1 -- 172.21.15.112:0/710369965 learned_addr learned my addr 172.21.15.112:0/710369965 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c1829640 1 -- 172.21.15.112:0/710369965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc06c630 msgr2=0x7f36bc0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c1829640 1 --2- 172.21.15.112:0/710369965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc06c630 0x7f36bc0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c1829640 1 -- 172.21.15.112:0/710369965 >> v1:172.21.15.187:6789/0 conn(0x7f36bc06ca10 legacy=0x7f36bc1477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.288+0000 7f36c1829640 1 -- 172.21.15.112:0/710369965 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc0afa00 con 0x7f36bc0add70 2024-04-25T00:55:51.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c1829640 1 --2- 172.21.15.112:0/710369965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc0add70 0x7f36bc0b4db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009e50 tx=0x7f36b401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6394a7c95ad6097d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:51.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36b3fff640 1 -- 172.21.15.112:0/710369965 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b40a6500 con 0x7f36bc0add70 2024-04-25T00:55:51.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36b3fff640 1 -- 172.21.15.112:0/710369965 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b40a66a0 con 0x7f36bc0add70 2024-04-25T00:55:51.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36b3fff640 1 -- 172.21.15.112:0/710369965 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b40a6a70 con 0x7f36bc0add70 2024-04-25T00:55:51.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 -- 172.21.15.112:0/710369965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc0add70 msgr2=0x7f36bc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 --2- 172.21.15.112:0/710369965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc0add70 0x7f36bc0b4db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009e50 tx=0x7f36b401c910 comp rx=0 tx=0).stop 2024-04-25T00:55:51.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 -- 172.21.15.112:0/710369965 shutdown_connections 2024-04-25T00:55:51.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 --2- 172.21.15.112:0/710369965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc0add70 0x7f36bc0b4db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 --2- 172.21.15.112:0/710369965 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc06c630 0x7f36bc0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 -- 172.21.15.112:0/710369965 >> 172.21.15.112:0/710369965 conn(0x7f36bc0b5980 msgr2=0x7f36bc0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:51.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 -- 172.21.15.112:0/710369965 shutdown_connections 2024-04-25T00:55:51.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 -- 172.21.15.112:0/710369965 wait complete. 2024-04-25T00:55:51.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 Processor -- start 2024-04-25T00:55:51.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.292+0000 7f36c328c640 1 -- start start 2024-04-25T00:55:51.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc06c630 0x7f36bc13e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc06ca10 0x7f36bc0b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc0add70 0x7f36bc1e0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1519f0 con 0x7f36bc06c630 2024-04-25T00:55:51.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc151870 con 0x7f36bc06ca10 2024-04-25T00:55:51.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f36bc151b70 con 0x7f36bc0add70 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc06c630 0x7f36bc13e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc06c630 0x7f36bc13e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51662/0 (socket says 172.21.15.112:51662) 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 -- 172.21.15.112:0/1509734632 learned_addr learned my addr 172.21.15.112:0/1509734632 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1829640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc0add70 0x7f36bc1e0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c0827640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc06ca10 0x7f36bc0b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 -- 172.21.15.112:0/1509734632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc0add70 msgr2=0x7f36bc1e0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc0add70 0x7f36bc1e0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 -- 172.21.15.112:0/1509734632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc06ca10 msgr2=0x7f36bc0b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc06ca10 0x7f36bc0b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 -- 172.21.15.112:0/1509734632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc13dea0 con 0x7f36bc06c630 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c1028640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc06c630 0x7f36bc13e660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e930 tx=0x7f36b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36b1ffb640 1 -- 172.21.15.112:0/1509734632 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8017070 con 0x7f36bc06c630 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36b1ffb640 1 -- 172.21.15.112:0/1509734632 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b800ca00 con 0x7f36bc06c630 2024-04-25T00:55:51.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc140460 con 0x7f36bc06c630 2024-04-25T00:55:51.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36b1ffb640 1 -- 172.21.15.112:0/1509734632 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b800cdb0 con 0x7f36bc06c630 2024-04-25T00:55:51.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc13f650 con 0x7f36bc06c630 2024-04-25T00:55:51.301 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc04f580 con 0x7f36bc06c630 2024-04-25T00:55:51.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36b1ffb640 1 -- 172.21.15.112:0/1509734632 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f36b801a070 con 0x7f36bc06c630 2024-04-25T00:55:51.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36b1ffb640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f36a00678c0 0x7f36a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:51.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c0827640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f36a00678c0 0x7f36a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:51.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36b1ffb640 1 -- 172.21.15.112:0/1509734632 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36b8085c20 con 0x7f36bc06c630 2024-04-25T00:55:51.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.296+0000 7f36c0827640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f36a00678c0 0x7f36a0069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f36bc140a40 tx=0x7f36ac007c50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:51.305 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.300+0000 7f36b1ffb640 1 -- 172.21.15.112:0/1509734632 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f36b80568b0 con 0x7f36bc06c630 2024-04-25T00:55:51.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36bc147510 con 0x7f36bc06c630 2024-04-25T00:55:51.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36b1ffb640 1 -- 172.21.15.112:0/1509734632 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f36b80561b0 con 0x7f36bc06c630 2024-04-25T00:55:51.487 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f36a00678c0 msgr2=0x7f36a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f36a00678c0 0x7f36a0069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f36bc140a40 tx=0x7f36ac007c50 comp rx=0 tx=0).stop 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc06c630 msgr2=0x7f36bc13e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc06c630 0x7f36bc13e660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e930 tx=0x7f36b800edf0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 shutdown_connections 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36bc0add70 0x7f36bc1e0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f36a00678c0 0x7f36a0069d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f36bc06ca10 0x7f36bc0b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 --2- 172.21.15.112:0/1509734632 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36bc06c630 0x7f36bc13e660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:51.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 >> 172.21.15.112:0/1509734632 conn(0x7f36bc0b5980 msgr2=0x7f36bc0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:51.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 shutdown_connections 2024-04-25T00:55:51.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51.484+0000 7f36c328c640 1 -- 172.21.15.112:0/1509734632 wait complete. 2024-04-25T00:55:51.504 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:55:51.505 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:55:51.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-25T00:55:51.505 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:55:51.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:55:51.505 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:55:51.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:55:51.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:55:51.505 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:55:51.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:51.505 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-25T00:55:51.511 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-25T00:55:51.512 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42517sec preferred_lft 42517sec 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 02:42:63:f8:7a:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:55:51.513 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:55:51.514 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:55:51.514 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:55:51.514 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T00:55:51.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:55:51.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2024-04-25T00:55:51.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2024-04-25T00:55:51.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:55:51.514 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T00:55:51.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:55:51.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:55:51.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:51.640 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:55:51.649 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-25T00:55:51.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:51.650 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:55:51.728 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:55:51.729 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:55:51.729 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T00:55:51.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:55:51.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:55:51.729 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:55:51.729 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T00:55:51.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:55:51.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:55:51.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:51.849 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T00:55:51.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:51.895 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2024-04-25T00:55:51.946 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:55:51.947 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T00:55:51.947 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:55:51.947 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T00:55:51.947 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T00:55:52.035 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:55:52.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:55:52.063 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T00:55:52.063 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:55:52.063 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T00:55:52.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T00:55:52.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:55:52.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T00:55:52.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T00:55:52.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T00:55:52.063 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T00:55:52.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:55:52.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:55:52.164 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:55:52.164 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T00:55:52.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-04-25T00:55:52.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:55:52.164 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T00:55:52.242 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:55:52.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:55:52.265 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:55:52.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:55:52.265 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:55:52.310 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:55:52.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:55:52.310 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:55:52.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-25T00:55:52.416 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-25T00:55:52.463 INFO:teuthology.orchestra.run.smithi112.stdout:/run/lock 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/fs/nfsd 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/run/rpc_pipefs 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-25T00:55:52.464 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-25T00:55:52.465 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:55:52.465 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:55:52.465 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:55:52.465 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:55:52.465 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-25T00:55:52.465 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:55:52.465 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:55:52.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:52.466 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-25T00:55:52.514 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:55:52.515 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:55:52.560 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-25T00:55:52.573 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:55:52.618 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-25T00:55:52.618 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-25T00:55:52.618 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/run/lock 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-25T00:55:52.619 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/fs/nfsd 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/run/rpc_pipefs 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:55:52.620 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:55:52.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:52.621 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-25T00:55:52.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:2024-04-25T00:55:52.636+0000 7f7bef255580 -1 init, newargv = 0x55df13ca37a0 newargc=15 2024-04-25T00:55:52.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[34807]: starting ceph client 2024-04-25T00:55:52.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[34807]: starting fuse 2024-04-25T00:55:53.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-25T00:55:53.757 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:55:53.805 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-25T00:55:53.805 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-25T00:55:53.805 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-25T00:55:53.805 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-25T00:55:53.805 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-25T00:55:53.805 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/run/lock 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/fs/nfsd 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-25T00:55:53.806 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/run/rpc_pipefs 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:55:53.807 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:55:53.808 INFO:teuthology.orchestra.run.smithi112.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:55:53.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:53.809 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-25T00:55:53.855 INFO:teuthology.orchestra.run.smithi112.stdout:45 2024-04-25T00:55:53.855 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> import re 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> import os 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T00:55:53.856 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> return f 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.0")) 2024-04-25T00:55:53.857 DEBUG:teuthology.orchestra.run.smithi112:> ' 2024-04-25T00:55:53.982 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.0.34807.asok 2024-04-25T00:55:53.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:55:53.991 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34807.asok 2024-04-25T00:55:53.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:53.991 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34807.asok status 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "0", 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "34807", 2024-04-25T00:55:54.147 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4581, 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4581 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:55:54.148 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3480289081 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3480289081 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.4581 192.168.0.1:0/3480289081", 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/3480289081", 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2024-04-25T00:55:54.149 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 8, 2024-04-25T00:55:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 20, 2024-04-25T00:55:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:55:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2024-04-25T00:55:54.150 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2024-04-25T00:55:54.150 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:55:54.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:54.161 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2024-04-25T00:55:54.161 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:55:54.167 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2024-04-25T00:55:54.167 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:55:54.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:55:54.167 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:55:54.222 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T00:55:54.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:54.386 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4373b67640 1 Processor -- start 2024-04-25T00:55:54.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4373b67640 1 -- start start 2024-04-25T00:55:54.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4373b67640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b23f0 0x7f436c0add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4373b67640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f436c0b3ce0 0x7f436c14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4373b67640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436c0aeec0 con 0x7f436c145200 2024-04-25T00:55:54.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4373b67640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f436c0af780 con 0x7f436c0b3ce0 2024-04-25T00:55:54.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4373b67640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f436c0b27d0 con 0x7f436c0b23f0 2024-04-25T00:55:54.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4371903640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b23f0 0x7f436c0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4372104640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f436c0b3ce0 0x7f436c14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4371903640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b23f0 0x7f436c0add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:52474/0 (socket says 172.21.15.112:52474) 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4371903640 1 -- 172.21.15.112:0/2242122761 learned_addr learned my addr 172.21.15.112:0/2242122761 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4370901640 1 -- 172.21.15.112:0/2242122761 <== mon.1 v1:172.21.15.187:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2658611716 0 0) 0x7f436c0aeec0 con 0x7f436c145200 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4371903640 1 -- 172.21.15.112:0/2242122761 >> v2:172.21.15.187:3300/0 conn(0x7f436c0b3ce0 msgr2=0x7f436c14fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4371903640 1 --2- 172.21.15.112:0/2242122761 >> v2:172.21.15.187:3300/0 conn(0x7f436c0b3ce0 0x7f436c14fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4371903640 1 -- 172.21.15.112:0/2242122761 >> v1:172.21.15.187:6789/0 conn(0x7f436c145200 legacy=0x7f436c072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.388+0000 7f4371903640 1 -- 172.21.15.112:0/2242122761 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c0b4150 con 0x7f436c0b23f0 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4371903640 1 --2- 172.21.15.112:0/2242122761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b23f0 0x7f436c0add70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4368009870 tx=0x7f436801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ae12eae6011249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4370901640 1 -- 172.21.15.112:0/2242122761 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680a7070 con 0x7f436c0b23f0 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4370901640 1 -- 172.21.15.112:0/2242122761 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43680b0070 con 0x7f436c0b23f0 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4370901640 1 -- 172.21.15.112:0/2242122761 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680ab890 con 0x7f436c0b23f0 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 -- 172.21.15.112:0/2242122761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b23f0 msgr2=0x7f436c0add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 --2- 172.21.15.112:0/2242122761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b23f0 0x7f436c0add70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4368009870 tx=0x7f436801c560 comp rx=0 tx=0).stop 2024-04-25T00:55:54.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 -- 172.21.15.112:0/2242122761 shutdown_connections 2024-04-25T00:55:54.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 --2- 172.21.15.112:0/2242122761 >> v2:172.21.15.187:3300/0 conn(0x7f436c0b3ce0 0x7f436c14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 --2- 172.21.15.112:0/2242122761 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b23f0 0x7f436c0add70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 -- 172.21.15.112:0/2242122761 >> 172.21.15.112:0/2242122761 conn(0x7f436c0b5980 msgr2=0x7f436c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:54.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 -- 172.21.15.112:0/2242122761 shutdown_connections 2024-04-25T00:55:54.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 -- 172.21.15.112:0/2242122761 wait complete. 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 Processor -- start 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.392+0000 7f4373b67640 1 -- start start 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f436c0b23f0 0x7f436c1428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b3ce0 0x7f436c1ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f436c145200 0x7f436c1de0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0b52d0 con 0x7f436c145200 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f436c0b5150 con 0x7f436c0b23f0 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f436c1528a0 con 0x7f436c0b3ce0 2024-04-25T00:55:54.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f436c145200 0x7f436c1de0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f436c145200 0x7f436c1de0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41776/0 (socket says 172.21.15.112:41776) 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 -- 172.21.15.112:0/1487326887 learned_addr learned my addr 172.21.15.112:0/1487326887 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4371903640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f436c0b23f0 0x7f436c1428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4371102640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b3ce0 0x7f436c1ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 -- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b3ce0 msgr2=0x7f436c1ddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b3ce0 0x7f436c1ddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 -- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f436c0b23f0 msgr2=0x7f436c1428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f436c0b23f0 0x7f436c1428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 -- 172.21.15.112:0/1487326887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c09cd50 con 0x7f436c145200 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4371903640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f436c0b23f0 0x7f436c1428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4371102640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b3ce0 0x7f436c1ddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4372104640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f436c145200 0x7f436c1de0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f436400efc0 tx=0x7f436400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:54.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f43627fc640 1 -- 172.21.15.112:0/1487326887 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364018020 con 0x7f436c145200 2024-04-25T00:55:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f43627fc640 1 -- 172.21.15.112:0/1487326887 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4364009b10 con 0x7f436c145200 2024-04-25T00:55:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436c09d030 con 0x7f436c145200 2024-04-25T00:55:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f43627fc640 1 -- 172.21.15.112:0/1487326887 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43640146c0 con 0x7f436c145200 2024-04-25T00:55:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436c09e310 con 0x7f436c145200 2024-04-25T00:55:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f436c04f580 con 0x7f436c145200 2024-04-25T00:55:54.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f43627fc640 1 -- 172.21.15.112:0/1487326887 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f436401b020 con 0x7f436c145200 2024-04-25T00:55:54.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f43627fc640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f43500678c0 0x7f4350069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f43627fc640 1 -- 172.21.15.112:0/1487326887 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4364086e30 con 0x7f436c145200 2024-04-25T00:55:54.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.396+0000 7f4371903640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f43500678c0 0x7f4350069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.400+0000 7f4371903640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f43500678c0 0x7f4350069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f43680af000 tx=0x7f43680a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:54.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.400+0000 7f43627fc640 1 -- 172.21.15.112:0/1487326887 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4364022030 con 0x7f436c145200 2024-04-25T00:55:54.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.560+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f436c0b84b0 con 0x7f436c145200 2024-04-25T00:55:54.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.564+0000 7f43627fc640 1 -- 172.21.15.112:0/1487326887 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f436c09cd50 con 0x7f436c145200 2024-04-25T00:55:54.565 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:54.565 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f43500678c0 msgr2=0x7f4350069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f43500678c0 0x7f4350069d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f43680af000 tx=0x7f43680a7000 comp rx=0 tx=0).stop 2024-04-25T00:55:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f436c145200 msgr2=0x7f436c1de0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f436c145200 0x7f436c1de0c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f436400efc0 tx=0x7f436400c490 comp rx=0 tx=0).stop 2024-04-25T00:55:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 shutdown_connections 2024-04-25T00:55:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f436c145200 0x7f436c1de0c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f436c0b3ce0 0x7f436c1ddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f43500678c0 0x7f4350069d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 --2- 172.21.15.112:0/1487326887 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f436c0b23f0 0x7f436c1428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 >> 172.21.15.112:0/1487326887 conn(0x7f436c0b5980 msgr2=0x7f436c0b7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 shutdown_connections 2024-04-25T00:55:54.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.568+0000 7f4373b67640 1 -- 172.21.15.112:0/1487326887 wait complete. 2024-04-25T00:55:54.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:54.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:54.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe0dc56640 1 Processor -- start 2024-04-25T00:55:54.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe0dc56640 1 -- start start 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe0dc56640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbe080b9030 0x7fbe0809acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe0dc56640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe080b9410 con 0x7fbe0814fcd0 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe0dc56640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe080baa90 con 0x7fbe0809b800 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe0dc56640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbe080b9990 con 0x7fbe080b9030 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe07fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbe0814fcd0 0x7fbe081520c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50304/0 (socket says 172.21.15.112:50304) 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe07fff640 1 -- 172.21.15.112:0/788317356 learned_addr learned my addr 172.21.15.112:0/788317356 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe077fe640 1 --2- 172.21.15.112:0/788317356 >> v2:172.21.15.187:3300/0 conn(0x7fbe080b9030 0x7fbe0809acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe067fc640 1 -- 172.21.15.112:0/788317356 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4082367772 0 0) 0x7fbe080b9410 con 0x7fbe0814fcd0 2024-04-25T00:55:54.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe067fc640 1 -- 172.21.15.112:0/788317356 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdf0003530 con 0x7fbe0814fcd0 2024-04-25T00:55:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe077fe640 1 -- 172.21.15.112:0/788317356 >> v1:172.21.15.187:6789/0 conn(0x7fbe0809b800 legacy=0x7fbe08141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe077fe640 1 -- 172.21.15.112:0/788317356 >> v1:172.21.15.112:6789/0 conn(0x7fbe0814fcd0 legacy=0x7fbe081520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.708+0000 7fbe077fe640 1 -- 172.21.15.112:0/788317356 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe080bad70 con 0x7fbe080b9030 2024-04-25T00:55:54.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe077fe640 1 --2- 172.21.15.112:0/788317356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe080b9030 0x7fbe0809acb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe000097c0 tx=0x7fbe0001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b89797dfee8c5149 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe067fc640 1 -- 172.21.15.112:0/788317356 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000b0070 con 0x7fbe080b9030 2024-04-25T00:55:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe067fc640 1 -- 172.21.15.112:0/788317356 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe000ab600 con 0x7fbe080b9030 2024-04-25T00:55:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe067fc640 1 -- 172.21.15.112:0/788317356 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000ab9b0 con 0x7fbe080b9030 2024-04-25T00:55:54.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- 172.21.15.112:0/788317356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe080b9030 msgr2=0x7fbe0809acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 --2- 172.21.15.112:0/788317356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe080b9030 0x7fbe0809acb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbe000097c0 tx=0x7fbe0001c410 comp rx=0 tx=0).stop 2024-04-25T00:55:54.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- 172.21.15.112:0/788317356 shutdown_connections 2024-04-25T00:55:54.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 --2- 172.21.15.112:0/788317356 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe080b9030 0x7fbe0809acb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- 172.21.15.112:0/788317356 >> 172.21.15.112:0/788317356 conn(0x7fbe0809eaf0 msgr2=0x7fbe0809ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:54.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- 172.21.15.112:0/788317356 shutdown_connections 2024-04-25T00:55:54.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- 172.21.15.112:0/788317356 wait complete. 2024-04-25T00:55:54.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 Processor -- start 2024-04-25T00:55:54.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- start start 2024-04-25T00:55:54.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0809b800 0x7fbe08152690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe080b9030 0x7fbe080b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0814fcd0 0x7fbe080b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0809b370 con 0x7fbe0814fcd0 2024-04-25T00:55:54.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0809b1f0 con 0x7fbe0809b800 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe0dc56640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbe08141a60 con 0x7fbe080b9030 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0814fcd0 0x7fbe080b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0814fcd0 0x7fbe080b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41792/0 (socket says 172.21.15.112:41792) 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 -- 172.21.15.112:0/2563756606 learned_addr learned my addr 172.21.15.112:0/2563756606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe077fe640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0809b800 0x7fbe08152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe06ffd640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe080b9030 0x7fbe080b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 -- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe080b9030 msgr2=0x7fbe080b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe080b9030 0x7fbe080b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 -- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0809b800 msgr2=0x7fbe08152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0809b800 0x7fbe08152690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 -- 172.21.15.112:0/2563756606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe081424c0 con 0x7fbe0814fcd0 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe06ffd640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe080b9030 0x7fbe080b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.712+0000 7fbe07fff640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0814fcd0 0x7fbe080b6980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800cbe0 tx=0x7fbdf800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:54.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbdeffff640 1 -- 172.21.15.112:0/2563756606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8017070 con 0x7fbe0814fcd0 2024-04-25T00:55:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbdeffff640 1 -- 172.21.15.112:0/2563756606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf800ad40 con 0x7fbe0814fcd0 2024-04-25T00:55:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbdeffff640 1 -- 172.21.15.112:0/2563756606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8012600 con 0x7fbe0814fcd0 2024-04-25T00:55:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe08143420 con 0x7fbe0814fcd0 2024-04-25T00:55:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe08141d10 con 0x7fbe0814fcd0 2024-04-25T00:55:54.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe0804f5f0 con 0x7fbe0814fcd0 2024-04-25T00:55:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbdeffff640 1 -- 172.21.15.112:0/2563756606 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbdf801a070 con 0x7fbe0814fcd0 2024-04-25T00:55:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbdeffff640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbdf40678f0 0x7fbdf4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbdeffff640 1 -- 172.21.15.112:0/2563756606 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbdf8085890 con 0x7fbe0814fcd0 2024-04-25T00:55:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbe077fe640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbdf40678f0 0x7fbdf4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:54.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.716+0000 7fbe077fe640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbdf40678f0 0x7fbdf4069db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe000ad000 tx=0x7fbe000a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:54.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.720+0000 7fbdeffff640 1 -- 172.21.15.112:0/2563756606 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbe081424c0 con 0x7fbe0814fcd0 2024-04-25T00:55:54.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.880+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe080b5cd0 con 0x7fbe0814fcd0 2024-04-25T00:55:54.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.880+0000 7fbdeffff640 1 -- 172.21.15.112:0/2563756606 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fbdf8055e10 con 0x7fbe0814fcd0 2024-04-25T00:55:54.881 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:54.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbdf40678f0 msgr2=0x7fbdf4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbdf40678f0 0x7fbdf4069db0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe000ad000 tx=0x7fbe000a6000 comp rx=0 tx=0).stop 2024-04-25T00:55:54.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0814fcd0 msgr2=0x7fbe080b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:54.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0814fcd0 0x7fbe080b6980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbdf800cbe0 tx=0x7fbdf800a430 comp rx=0 tx=0).stop 2024-04-25T00:55:54.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 shutdown_connections 2024-04-25T00:55:54.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe0814fcd0 0x7fbe080b6980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbe080b9030 0x7fbe080b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbdf40678f0 0x7fbdf4069db0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 --2- 172.21.15.112:0/2563756606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbe0809b800 0x7fbe08152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:54.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 >> 172.21.15.112:0/2563756606 conn(0x7fbe0809eaf0 msgr2=0x7fbe080b8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:54.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 shutdown_connections 2024-04-25T00:55:54.885 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:54.884+0000 7fbe0dc56640 1 -- 172.21.15.112:0/2563756606 wait complete. 2024-04-25T00:55:54.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:55:55.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.016+0000 7f9e8189d640 1 Processor -- start 2024-04-25T00:55:55.018 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.016+0000 7f9e8189d640 1 -- start start 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e8189d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c147900 0x7f9e7c147d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e8189d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9e7c1482a0 0x7f9e7c14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e8189d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e7c148730 con 0x7f9e7c06c170 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e8189d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9e7c0ae8a0 con 0x7f9e7c1482a0 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e8189d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c0af4e0 con 0x7f9e7c147900 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7a7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c147900 0x7f9e7c147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7b7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9e7c1482a0 0x7f9e7c14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7a7fc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c147900 0x7f9e7c147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:52504/0 (socket says 172.21.15.112:52504) 2024-04-25T00:55:55.019 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7a7fc640 1 -- 172.21.15.112:0/3194291008 learned_addr learned my addr 172.21.15.112:0/3194291008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:55.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7a7fc640 1 -- 172.21.15.112:0/3194291008 >> v2:172.21.15.187:3300/0 conn(0x7f9e7c1482a0 msgr2=0x7f9e7c14ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:55.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7a7fc640 1 --2- 172.21.15.112:0/3194291008 >> v2:172.21.15.187:3300/0 conn(0x7f9e7c1482a0 0x7f9e7c14ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7a7fc640 1 -- 172.21.15.112:0/3194291008 >> v1:172.21.15.187:6789/0 conn(0x7f9e7c06c170 legacy=0x7f9e7c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7a7fc640 1 -- 172.21.15.112:0/3194291008 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c06c550 con 0x7f9e7c147900 2024-04-25T00:55:55.020 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7b7fe640 1 --2- 172.21.15.112:0/3194291008 >> v2:172.21.15.187:3300/0 conn(0x7f9e7c1482a0 0x7f9e7c14ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:55:55.021 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e7a7fc640 1 --2- 172.21.15.112:0/3194291008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c147900 0x7f9e7c147d60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e640098a0 tx=0x7f9e6401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db7fe936e754008a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e79ffb640 1 -- 172.21.15.112:0/3194291008 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640b1070 con 0x7f9e7c147900 2024-04-25T00:55:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e79ffb640 1 -- 172.21.15.112:0/3194291008 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e640ac600 con 0x7f9e7c147900 2024-04-25T00:55:55.022 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.020+0000 7f9e79ffb640 1 -- 172.21.15.112:0/3194291008 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e640ac9d0 con 0x7f9e7c147900 2024-04-25T00:55:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- 172.21.15.112:0/3194291008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c147900 msgr2=0x7f9e7c147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 --2- 172.21.15.112:0/3194291008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c147900 0x7f9e7c147d60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9e640098a0 tx=0x7f9e6401c4b0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- 172.21.15.112:0/3194291008 shutdown_connections 2024-04-25T00:55:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 --2- 172.21.15.112:0/3194291008 >> v2:172.21.15.187:3300/0 conn(0x7f9e7c1482a0 0x7f9e7c14ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 --2- 172.21.15.112:0/3194291008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c147900 0x7f9e7c147d60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- 172.21.15.112:0/3194291008 >> 172.21.15.112:0/3194291008 conn(0x7f9e7c0b8c80 msgr2=0x7f9e7c0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- 172.21.15.112:0/3194291008 shutdown_connections 2024-04-25T00:55:55.023 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- 172.21.15.112:0/3194291008 wait complete. 2024-04-25T00:55:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 Processor -- start 2024-04-25T00:55:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- start start 2024-04-25T00:55:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c06c170 0x7f9e7c09c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e7c147900 0x7f9e7c143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7c1482a0 0x7f9e7c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c0adef0 con 0x7f9e7c1482a0 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7c0ae530 con 0x7f9e7c147900 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9e7c0b2e80 con 0x7f9e7c06c170 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7c1482a0 0x7f9e7c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7c1482a0 0x7f9e7c06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41806/0 (socket says 172.21.15.112:41806) 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 -- 172.21.15.112:0/562890201 learned_addr learned my addr 172.21.15.112:0/562890201 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7a7fc640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e7c147900 0x7f9e7c143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7affd640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c06c170 0x7f9e7c09c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 -- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c06c170 msgr2=0x7f9e7c09c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c06c170 0x7f9e7c09c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 -- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e7c147900 msgr2=0x7f9e7c143570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.026 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e7c147900 0x7f9e7c143570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 -- 172.21.15.112:0/562890201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7c143c30 con 0x7f9e7c1482a0 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7a7fc640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e7c147900 0x7f9e7c143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e7b7fe640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7c1482a0 0x7f9e7c06f5e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00e930 tx=0x7f9e6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e73fff640 1 -- 172.21.15.112:0/562890201 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c017070 con 0x7f9e7c1482a0 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e73fff640 1 -- 172.21.15.112:0/562890201 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e6c004020 con 0x7f9e7c1482a0 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e73fff640 1 -- 172.21.15.112:0/562890201 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c004430 con 0x7f9e7c1482a0 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7c143f10 con 0x7f9e7c1482a0 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e7c071c00 con 0x7f9e7c1482a0 2024-04-25T00:55:55.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.024+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e7c071a10 con 0x7f9e7c1482a0 2024-04-25T00:55:55.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.028+0000 7f9e73fff640 1 -- 172.21.15.112:0/562890201 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e6c00c8e0 con 0x7f9e7c1482a0 2024-04-25T00:55:55.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.028+0000 7f9e73fff640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e5c0678c0 0x7f9e5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.028+0000 7f9e7affd640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e5c0678c0 0x7f9e5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.028+0000 7f9e73fff640 1 -- 172.21.15.112:0/562890201 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f9e6c0859d0 con 0x7f9e7c1482a0 2024-04-25T00:55:55.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.028+0000 7f9e7affd640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e5c0678c0 0x7f9e5c069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e74001d50 tx=0x7f9e74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.032+0000 7f9e73fff640 1 -- 172.21.15.112:0/562890201 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e7c143c30 con 0x7f9e7c1482a0 2024-04-25T00:55:55.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.176+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9e7c04f5f0 con 0x7f9e7c1482a0 2024-04-25T00:55:55.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e73fff640 1 -- 172.21.15.112:0/562890201 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f9e6c0565a0 con 0x7f9e7c1482a0 2024-04-25T00:55:55.178 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e5c0678c0 msgr2=0x7f9e5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e5c0678c0 0x7f9e5c069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9e74001d50 tx=0x7f9e74009040 comp rx=0 tx=0).stop 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7c1482a0 msgr2=0x7f9e7c06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7c1482a0 0x7f9e7c06f5e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c00e930 tx=0x7f9e6c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 shutdown_connections 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e7c1482a0 0x7f9e7c06f5e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9e5c0678c0 0x7f9e5c069d80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9e7c147900 0x7f9e7c143570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 --2- 172.21.15.112:0/562890201 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9e7c06c170 0x7f9e7c09c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 >> 172.21.15.112:0/562890201 conn(0x7f9e7c0b8c80 msgr2=0x7f9e7c0b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:55.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 shutdown_connections 2024-04-25T00:55:55.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.180+0000 7f9e8189d640 1 -- 172.21.15.112:0/562890201 wait complete. 2024-04-25T00:55:55.197 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:55:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.312+0000 7f8dbd523640 1 Processor -- start 2024-04-25T00:55:55.313 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.312+0000 7f8dbd523640 1 -- start start 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 0x7f8db809da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8db809dfa0 0x7f8db8148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f8db80b0390 0x7f8db8148d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f8db80b1d70 con 0x7f8db809dfa0 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f8db80b1600 con 0x7f8db80b0390 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8db80b1390 con 0x7f8db80affb0 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8db809dfa0 0x7f8db8148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8db809dfa0 0x7f8db8148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41816/0 (socket says 172.21.15.112:41816) 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 -- 172.21.15.112:0/3613465845 learned_addr learned my addr 172.21.15.112:0/3613465845 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:55.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db6ffd640 1 --2- 172.21.15.112:0/3613465845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 0x7f8db809da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db77fe640 1 --2- 172.21.15.112:0/3613465845 >> v2:172.21.15.187:3300/0 conn(0x7f8db80b0390 0x7f8db8148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 -- 172.21.15.112:0/3613465845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 msgr2=0x7f8db809da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 --2- 172.21.15.112:0/3613465845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 0x7f8db809da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 -- 172.21.15.112:0/3613465845 >> v2:172.21.15.187:3300/0 conn(0x7f8db80b0390 msgr2=0x7f8db8148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 --2- 172.21.15.112:0/3613465845 >> v2:172.21.15.187:3300/0 conn(0x7f8db80b0390 0x7f8db8148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 -- 172.21.15.112:0/3613465845 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db80b1fa0 con 0x7f8db809dfa0 2024-04-25T00:55:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db67fc640 1 --2- 172.21.15.112:0/3613465845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db8148800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8db8072ab0 tx=0x7f8dac01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1897ff64e3a0dd13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db5ffb640 1 -- 172.21.15.112:0/3613465845 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0ab430 con 0x7f8db809dfa0 2024-04-25T00:55:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db5ffb640 1 -- 172.21.15.112:0/3613465845 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dac0b0070 con 0x7f8db809dfa0 2024-04-25T00:55:55.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8db5ffb640 1 -- 172.21.15.112:0/3613465845 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dac0ab7f0 con 0x7f8db809dfa0 2024-04-25T00:55:55.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 -- 172.21.15.112:0/3613465845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 msgr2=0x7f8db8148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 --2- 172.21.15.112:0/3613465845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db8148800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8db8072ab0 tx=0x7f8dac01c470 comp rx=0 tx=0).stop 2024-04-25T00:55:55.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 -- 172.21.15.112:0/3613465845 shutdown_connections 2024-04-25T00:55:55.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 --2- 172.21.15.112:0/3613465845 >> v2:172.21.15.187:3300/0 conn(0x7f8db80b0390 0x7f8db8148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 --2- 172.21.15.112:0/3613465845 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db8148800 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 --2- 172.21.15.112:0/3613465845 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 0x7f8db809da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 -- 172.21.15.112:0/3613465845 >> 172.21.15.112:0/3613465845 conn(0x7f8db80b8c80 msgr2=0x7f8db80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:55.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 -- 172.21.15.112:0/3613465845 shutdown_connections 2024-04-25T00:55:55.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.316+0000 7f8dbd523640 1 -- 172.21.15.112:0/3613465845 wait complete. 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 Processor -- start 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 -- start start 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 0x7f8db809b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8db80b0390 0x7f8db81e0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8db8145b20 con 0x7f8db809dfa0 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8db81459a0 con 0x7f8db80b0390 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8db8145ca0 con 0x7f8db80affb0 2024-04-25T00:55:55.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db80b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41830/0 (socket says 172.21.15.112:41830) 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 -- 172.21.15.112:0/1572754974 learned_addr learned my addr 172.21.15.112:0/1572754974 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db67fc640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 0x7f8db809b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db77fe640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8db80b0390 0x7f8db81e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 -- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 msgr2=0x7f8db809b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 0x7f8db809b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 -- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8db80b0390 msgr2=0x7f8db81e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8db80b0390 0x7f8db81e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 -- 172.21.15.112:0/1572754974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db81e0440 con 0x7f8db809dfa0 2024-04-25T00:55:55.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db77fe640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8db80b0390 0x7f8db81e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:55.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8db6ffd640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db80b4450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8da800e9b0 tx=0x7f8da800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/1572754974 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8017070 con 0x7f8db809dfa0 2024-04-25T00:55:55.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/1572754974 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da800c9a0 con 0x7f8db809dfa0 2024-04-25T00:55:55.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/1572754974 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da800cd70 con 0x7f8db809dfa0 2024-04-25T00:55:55.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db809abd0 con 0x7f8db809dfa0 2024-04-25T00:55:55.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db8142df0 con 0x7f8db809dfa0 2024-04-25T00:55:55.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db8068470 con 0x7f8db809dfa0 2024-04-25T00:55:55.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.320+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/1572754974 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8da801a070 con 0x7f8db809dfa0 2024-04-25T00:55:55.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.324+0000 7f8d9f7fe640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8da40678c0 0x7f8da4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.324+0000 7f8db67fc640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8da40678c0 0x7f8da4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.324+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/1572754974 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8da8085cc0 con 0x7f8db809dfa0 2024-04-25T00:55:55.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.324+0000 7f8db67fc640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8da40678c0 0x7f8da4069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0af000 tx=0x7f8dac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.328+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/1572754974 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8da8056890 con 0x7f8db809dfa0 2024-04-25T00:55:55.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.480+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8db80b1040 con 0x7f8db809dfa0 2024-04-25T00:55:55.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.480+0000 7f8d9f7fe640 1 -- 172.21.15.112:0/1572754974 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f8da8056190 con 0x7f8db809dfa0 2024-04-25T00:55:55.480 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:55:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8da40678c0 msgr2=0x7f8da4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8da40678c0 0x7f8da4069d80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0af000 tx=0x7f8dac008040 comp rx=0 tx=0).stop 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 msgr2=0x7f8db80b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db80b4450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8da800e9b0 tx=0x7f8da800ee70 comp rx=0 tx=0).stop 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 shutdown_connections 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8da40678c0 0x7f8da4069d80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8db80b0390 0x7f8db81e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8db80affb0 0x7f8db809b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 --2- 172.21.15.112:0/1572754974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8db809dfa0 0x7f8db80b4450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 >> 172.21.15.112:0/1572754974 conn(0x7f8db80b8c80 msgr2=0x7f8db806bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 shutdown_connections 2024-04-25T00:55:55.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.484+0000 7f8dbd523640 1 -- 172.21.15.112:0/1572754974 wait complete. 2024-04-25T00:55:55.498 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.604+0000 7f6c7bb19640 1 Processor -- start 2024-04-25T00:55:55.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.604+0000 7f6c7bb19640 1 -- start start 2024-04-25T00:55:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.604+0000 7f6c7bb19640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c740add70 0x7f6c74145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.604+0000 7f6c7bb19640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 0x7f6c7414fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.604+0000 7f6c7bb19640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c740aeec0 con 0x7f6c740b23f0 2024-04-25T00:55:55.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.604+0000 7f6c7bb19640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6c740af780 con 0x7f6c740add70 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.604+0000 7f6c7bb19640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6c740b27d0 con 0x7f6c740b3ce0 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c790b4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c740add70 0x7f6c74145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c790b4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6c740add70 0x7f6c74145200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41838/0 (socket says 172.21.15.112:41838) 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c790b4640 1 -- 172.21.15.112:0/3113736841 learned_addr learned my addr 172.21.15.112:0/3113736841 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7a0b6640 1 --2- 172.21.15.112:0/3113736841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 0x7f6c7414fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c790b4640 1 -- 172.21.15.112:0/3113736841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 msgr2=0x7f6c7414fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c790b4640 1 --2- 172.21.15.112:0/3113736841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 0x7f6c7414fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c790b4640 1 -- 172.21.15.112:0/3113736841 >> v1:172.21.15.112:6789/0 conn(0x7f6c740b23f0 legacy=0x7f6c74072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c790b4640 1 -- 172.21.15.112:0/3113736841 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c740b4150 con 0x7f6c740add70 2024-04-25T00:55:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c790b4640 1 --2- 172.21.15.112:0/3113736841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740add70 0x7f6c74145200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009870 tx=0x7f6c6401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e544b8f4713dee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c788b3640 1 -- 172.21.15.112:0/3113736841 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c640a9620 con 0x7f6c740add70 2024-04-25T00:55:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c788b3640 1 -- 172.21.15.112:0/3113736841 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c640a97c0 con 0x7f6c740add70 2024-04-25T00:55:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c788b3640 1 -- 172.21.15.112:0/3113736841 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c640a9b90 con 0x7f6c740add70 2024-04-25T00:55:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7bb19640 1 -- 172.21.15.112:0/3113736841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740add70 msgr2=0x7f6c74145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/3113736841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740add70 0x7f6c74145200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009870 tx=0x7f6c6401c560 comp rx=0 tx=0).stop 2024-04-25T00:55:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7bb19640 1 -- 172.21.15.112:0/3113736841 shutdown_connections 2024-04-25T00:55:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/3113736841 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 0x7f6c7414fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/3113736841 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740add70 0x7f6c74145200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7bb19640 1 -- 172.21.15.112:0/3113736841 >> 172.21.15.112:0/3113736841 conn(0x7f6c740b5980 msgr2=0x7f6c740b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7bb19640 1 -- 172.21.15.112:0/3113736841 shutdown_connections 2024-04-25T00:55:55.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.608+0000 7f6c7bb19640 1 -- 172.21.15.112:0/3113736841 wait complete. 2024-04-25T00:55:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 Processor -- start 2024-04-25T00:55:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 -- start start 2024-04-25T00:55:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740add70 0x7f6c74138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740b23f0 0x7f6c7413f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 0x7f6c7413d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6c740b52d0 con 0x7f6c740b23f0 2024-04-25T00:55:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6c740b5150 con 0x7f6c740add70 2024-04-25T00:55:55.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6c741528a0 con 0x7f6c740b3ce0 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740b23f0 0x7f6c7413f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740b23f0 0x7f6c7413f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41846/0 (socket says 172.21.15.112:41846) 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 -- 172.21.15.112:0/4095436182 learned_addr learned my addr 172.21.15.112:0/4095436182 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7a0b6640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 0x7f6c7413d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 -- 172.21.15.112:0/4095436182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 msgr2=0x7f6c7413d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 0x7f6c7413d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 -- 172.21.15.112:0/4095436182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740add70 msgr2=0x7f6c74138e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740add70 0x7f6c74138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 -- 172.21.15.112:0/4095436182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c74071100 con 0x7f6c740b23f0 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c790b4640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740b23f0 0x7f6c7413f050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6c640ab040 tx=0x7f6c6401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c6a7fc640 1 -- 172.21.15.112:0/4095436182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c640bc450 con 0x7f6c740b23f0 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c6a7fc640 1 -- 172.21.15.112:0/4095436182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c640bc610 con 0x7f6c740b23f0 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c6a7fc640 1 -- 172.21.15.112:0/4095436182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c640bca00 con 0x7f6c740b23f0 2024-04-25T00:55:55.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c740713e0 con 0x7f6c740b23f0 2024-04-25T00:55:55.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c74142db0 con 0x7f6c740b23f0 2024-04-25T00:55:55.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7404f580 con 0x7f6c740b23f0 2024-04-25T00:55:55.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c6a7fc640 1 -- 172.21.15.112:0/4095436182 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6c640b13f0 con 0x7f6c740b23f0 2024-04-25T00:55:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.612+0000 7f6c6a7fc640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6c580678c0 0x7f6c58069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.616+0000 7f6c798b5640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6c580678c0 0x7f6c58069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.616+0000 7f6c6a7fc640 1 -- 172.21.15.112:0/4095436182 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6c64120300 con 0x7f6c740b23f0 2024-04-25T00:55:55.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.616+0000 7f6c798b5640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6c580678c0 0x7f6c58069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c70000f70 tx=0x7f6c700079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.620+0000 7f6c6a7fc640 1 -- 172.21.15.112:0/4095436182 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6c640ad020 con 0x7f6c740b23f0 2024-04-25T00:55:55.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.772+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c74139910 con 0x7f6c740b23f0 2024-04-25T00:55:55.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c6a7fc640 1 -- 172.21.15.112:0/4095436182 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f6c64016150 con 0x7f6c740b23f0 2024-04-25T00:55:55.774 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:55.775 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6c580678c0 msgr2=0x7f6c58069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6c580678c0 0x7f6c58069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c70000f70 tx=0x7f6c700079e0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740b23f0 msgr2=0x7f6c7413f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740b23f0 0x7f6c7413f050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6c640ab040 tx=0x7f6c6401ada0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 shutdown_connections 2024-04-25T00:55:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6c740b3ce0 0x7f6c7413d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6c740b23f0 0x7f6c7413f050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6c580678c0 0x7f6c58069d80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 --2- 172.21.15.112:0/4095436182 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6c740add70 0x7f6c74138e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 >> 172.21.15.112:0/4095436182 conn(0x7f6c740b5980 msgr2=0x7f6c7406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 shutdown_connections 2024-04-25T00:55:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.776+0000 7f6c7bb19640 1 -- 172.21.15.112:0/4095436182 wait complete. 2024-04-25T00:55:55.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:55.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:55.917 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.916+0000 7fa5bf37f640 1 Processor -- start 2024-04-25T00:55:55.918 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.916+0000 7fa5bf37f640 1 -- start start 2024-04-25T00:55:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bf37f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa5b8147900 0x7fa5b8147d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bf37f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5b81482a0 0x7fa5b814ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bf37f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b80af4e0 con 0x7fa5b806c170 2024-04-25T00:55:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bf37f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b80ae8a0 con 0x7fa5b81482a0 2024-04-25T00:55:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bf37f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b806c550 con 0x7fa5b8147900 2024-04-25T00:55:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bd11b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa5b806c170 0x7fa5b80aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50314/0 (socket says 172.21.15.112:50314) 2024-04-25T00:55:55.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bd91c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5b81482a0 0x7fa5b814ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bd11b640 1 -- 172.21.15.112:0/2642103380 learned_addr learned my addr 172.21.15.112:0/2642103380 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bc91a640 1 --2- 172.21.15.112:0/2642103380 >> v2:172.21.15.187:3300/0 conn(0x7fa5b8147900 0x7fa5b8147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5b7fff640 1 -- 172.21.15.112:0/2642103380 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4284289978 0 0) 0x7fa5b80af4e0 con 0x7fa5b806c170 2024-04-25T00:55:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5b7fff640 1 -- 172.21.15.112:0/2642103380 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5a0005740 con 0x7fa5b806c170 2024-04-25T00:55:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bd91c640 1 -- 172.21.15.112:0/2642103380 >> v2:172.21.15.187:3300/0 conn(0x7fa5b8147900 msgr2=0x7fa5b8147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bd91c640 1 --2- 172.21.15.112:0/2642103380 >> v2:172.21.15.187:3300/0 conn(0x7fa5b8147900 0x7fa5b8147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bd91c640 1 -- 172.21.15.112:0/2642103380 >> v1:172.21.15.112:6789/0 conn(0x7fa5b806c170 legacy=0x7fa5b80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.920+0000 7fa5bd91c640 1 -- 172.21.15.112:0/2642103380 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b80ae220 con 0x7fa5b81482a0 2024-04-25T00:55:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 --2- 172.21.15.112:0/2642103380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b814ed60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80b4b10 tx=0x7fa5b001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb08dac998b48b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5b7fff640 1 -- 172.21.15.112:0/2642103380 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00b0070 con 0x7fa5b81482a0 2024-04-25T00:55:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5b7fff640 1 -- 172.21.15.112:0/2642103380 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b00ab600 con 0x7fa5b81482a0 2024-04-25T00:55:55.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5b7fff640 1 -- 172.21.15.112:0/2642103380 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00ab9d0 con 0x7fa5b81482a0 2024-04-25T00:55:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- 172.21.15.112:0/2642103380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 msgr2=0x7fa5b814ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/2642103380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b814ed60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa5b80b4b10 tx=0x7fa5b001c3f0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- 172.21.15.112:0/2642103380 shutdown_connections 2024-04-25T00:55:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/2642103380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b814ed60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/2642103380 >> v2:172.21.15.187:3300/0 conn(0x7fa5b8147900 0x7fa5b8147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- 172.21.15.112:0/2642103380 >> 172.21.15.112:0/2642103380 conn(0x7fa5b80b8c80 msgr2=0x7fa5b80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- 172.21.15.112:0/2642103380 shutdown_connections 2024-04-25T00:55:55.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- 172.21.15.112:0/2642103380 wait complete. 2024-04-25T00:55:55.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 Processor -- start 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- start start 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b806c170 0x7fa5b813f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b8147900 0x7fa5b8144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b8139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b80adef0 con 0x7fa5b81482a0 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b80ae530 con 0x7fa5b806c170 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b80b2e80 con 0x7fa5b8147900 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b8139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b8139c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41870/0 (socket says 172.21.15.112:41870) 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 -- 172.21.15.112:0/3115290840 learned_addr learned my addr 172.21.15.112:0/3115290840 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:55.926 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd11b640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b806c170 0x7fa5b813f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bc91a640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b8147900 0x7fa5b8144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 -- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b8147900 msgr2=0x7fa5b8144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b8147900 0x7fa5b8144d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 -- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b806c170 msgr2=0x7fa5b813f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b806c170 0x7fa5b813f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 -- 172.21.15.112:0/3115290840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b81db750 con 0x7fa5b81482a0 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bc91a640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b8147900 0x7fa5b8144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bd91c640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b8139c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00ae040 tx=0x7fa5b0019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5b5ffb640 1 -- 172.21.15.112:0/3115290840 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00b8020 con 0x7fa5b81482a0 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5b5ffb640 1 -- 172.21.15.112:0/3115290840 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5b00b4630 con 0x7fa5b81482a0 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b81dba30 con 0x7fa5b81482a0 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5b5ffb640 1 -- 172.21.15.112:0/3115290840 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00b0040 con 0x7fa5b81482a0 2024-04-25T00:55:55.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.924+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b813fe70 con 0x7fa5b81482a0 2024-04-25T00:55:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.928+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b804f5f0 con 0x7fa5b81482a0 2024-04-25T00:55:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.928+0000 7fa5b5ffb640 1 -- 172.21.15.112:0/3115290840 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa5b00bb020 con 0x7fa5b81482a0 2024-04-25T00:55:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.928+0000 7fa5b5ffb640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5a40678c0 0x7fa5a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:55.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.928+0000 7fa5b5ffb640 1 -- 172.21.15.112:0/3115290840 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa5b011f9a0 con 0x7fa5b81482a0 2024-04-25T00:55:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.928+0000 7fa5bd11b640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5a40678c0 0x7fa5a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:55.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.928+0000 7fa5bd11b640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5a40678c0 0x7fa5a4069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8005c10 tx=0x7fa5a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:55.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:55.932+0000 7fa5b5ffb640 1 -- 172.21.15.112:0/3115290840 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa5b00c2030 con 0x7fa5b81482a0 2024-04-25T00:55:56.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.092+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa5b80b68b0 con 0x7fa5b81482a0 2024-04-25T00:55:56.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.092+0000 7fa5b5ffb640 1 -- 172.21.15.112:0/3115290840 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa5b0016070 con 0x7fa5b81482a0 2024-04-25T00:55:56.093 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:56.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5a40678c0 msgr2=0x7fa5a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5a40678c0 0x7fa5a4069d80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5a8005c10 tx=0x7fa5a8009040 comp rx=0 tx=0).stop 2024-04-25T00:55:56.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 msgr2=0x7fa5b8139c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b8139c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00ae040 tx=0x7fa5b0019190 comp rx=0 tx=0).stop 2024-04-25T00:55:56.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 shutdown_connections 2024-04-25T00:55:56.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5b81482a0 0x7fa5b8139c70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5b8147900 0x7fa5b8144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5a40678c0 0x7fa5a4069d80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 --2- 172.21.15.112:0/3115290840 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa5b806c170 0x7fa5b813f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 >> 172.21.15.112:0/3115290840 conn(0x7fa5b80b8c80 msgr2=0x7fa5b80b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 shutdown_connections 2024-04-25T00:55:56.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.096+0000 7fa5bf37f640 1 -- 172.21.15.112:0/3115290840 wait complete. 2024-04-25T00:55:56.112 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:55:56.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb8621640 1 Processor -- start 2024-04-25T00:55:56.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb8621640 1 -- start start 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb8621640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbbb00b9c60 0x7fbbb00b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb8621640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fbbb01473f0 0x7fbbb0147880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb8621640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbb0147dc0 con 0x7fbbb00b5800 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb8621640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb00b6b70 con 0x7fbbb00b9c60 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb8621640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fbbb00b4200 con 0x7fbbb01473f0 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb63bd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbbb00b9c60 0x7fbbb00b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb5bbc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbbb00b5800 0x7fbbb0140f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50316/0 (socket says 172.21.15.112:50316) 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb5bbc640 1 -- 172.21.15.112:0/2260472206 learned_addr learned my addr 172.21.15.112:0/2260472206 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb63bd640 1 -- 172.21.15.112:0/2260472206 >> v2:172.21.15.187:3300/0 conn(0x7fbbb01473f0 msgr2=0x7fbbb0147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb6bbe640 1 --2- 172.21.15.112:0/2260472206 >> v2:172.21.15.187:3300/0 conn(0x7fbbb01473f0 0x7fbbb0147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb63bd640 1 --2- 172.21.15.112:0/2260472206 >> v2:172.21.15.187:3300/0 conn(0x7fbbb01473f0 0x7fbbb0147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb63bd640 1 -- 172.21.15.112:0/2260472206 >> v1:172.21.15.112:6789/0 conn(0x7fbbb00b5800 legacy=0x7fbbb0140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb53bb640 1 -- 172.21.15.112:0/2260472206 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1396283116 0 0) 0x7fbbb0147dc0 con 0x7fbbb00b5800 2024-04-25T00:55:56.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb63bd640 1 -- 172.21.15.112:0/2260472206 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb00b44e0 con 0x7fbbb00b9c60 2024-04-25T00:55:56.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.232+0000 7fbbb63bd640 1 --2- 172.21.15.112:0/2260472206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b9c60 0x7fbbb00b52c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00b7120 tx=0x7fbbac01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19f53bdbfeae040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb53bb640 1 -- 172.21.15.112:0/2260472206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b0070 con 0x7fbbb00b9c60 2024-04-25T00:55:56.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb53bb640 1 -- 172.21.15.112:0/2260472206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac0ab600 con 0x7fbbb00b9c60 2024-04-25T00:55:56.234 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb53bb640 1 -- 172.21.15.112:0/2260472206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0ab9d0 con 0x7fbbb00b9c60 2024-04-25T00:55:56.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- 172.21.15.112:0/2260472206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b9c60 msgr2=0x7fbbb00b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2260472206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b9c60 0x7fbbb00b52c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbbb00b7120 tx=0x7fbbac01c360 comp rx=0 tx=0).stop 2024-04-25T00:55:56.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- 172.21.15.112:0/2260472206 shutdown_connections 2024-04-25T00:55:56.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2260472206 >> v2:172.21.15.187:3300/0 conn(0x7fbbb01473f0 0x7fbbb0147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2260472206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b9c60 0x7fbbb00b52c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- 172.21.15.112:0/2260472206 >> 172.21.15.112:0/2260472206 conn(0x7fbbb0067120 msgr2=0x7fbbb0067530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:56.235 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- 172.21.15.112:0/2260472206 shutdown_connections 2024-04-25T00:55:56.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- 172.21.15.112:0/2260472206 wait complete. 2024-04-25T00:55:56.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 Processor -- start 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- start start 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b5800 0x7fbbb00bc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbb00b9c60 0x7fbbb0136220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbb01473f0 0x7fbbb01e0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0154610 con 0x7fbbb00b5800 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0154490 con 0x7fbbb01473f0 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb0154790 con 0x7fbbb00b9c60 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b5800 0x7fbbb00bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b5800 0x7fbbb00bc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41880/0 (socket says 172.21.15.112:41880) 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 -- 172.21.15.112:0/2611101701 learned_addr learned my addr 172.21.15.112:0/2611101701 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 -- 172.21.15.112:0/2611101701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbb00b9c60 msgr2=0x7fbbb0136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb5bbc640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbb00b9c60 0x7fbbb0136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbb00b9c60 0x7fbbb0136220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 -- 172.21.15.112:0/2611101701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbb01473f0 msgr2=0x7fbbb01e0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbb01473f0 0x7fbbb01e0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 -- 172.21.15.112:0/2611101701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb0136a70 con 0x7fbbb00b5800 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb5bbc640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbb00b9c60 0x7fbbb0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb63bd640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b5800 0x7fbbb00bc410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0ae040 tx=0x7fbbac019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbba6ffd640 1 -- 172.21.15.112:0/2611101701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b8020 con 0x7fbbb00b5800 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbba6ffd640 1 -- 172.21.15.112:0/2611101701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbac0b4610 con 0x7fbbb00b5800 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb0136760 con 0x7fbbb00b5800 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbba6ffd640 1 -- 172.21.15.112:0/2611101701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b0040 con 0x7fbbb00b5800 2024-04-25T00:55:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.236+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01e0a70 con 0x7fbbb00b5800 2024-04-25T00:55:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.240+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb01327b0 con 0x7fbbb00b5800 2024-04-25T00:55:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.240+0000 7fbba6ffd640 1 -- 172.21.15.112:0/2611101701 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbbac0bb020 con 0x7fbbb00b5800 2024-04-25T00:55:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.240+0000 7fbba6ffd640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbb940678c0 0x7fbb94069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.240+0000 7fbba6ffd640 1 -- 172.21.15.112:0/2611101701 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbbac11f990 con 0x7fbbb00b5800 2024-04-25T00:55:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.240+0000 7fbbb5bbc640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbb940678c0 0x7fbb94069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.240 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.240+0000 7fbbb5bbc640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbb940678c0 0x7fbb94069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbbb01386e0 tx=0x7fbba0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.243 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.244+0000 7fbba6ffd640 1 -- 172.21.15.112:0/2611101701 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbbac0c2030 con 0x7fbbb00b5800 2024-04-25T00:55:56.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbbb00b9010 con 0x7fbbb00b5800 2024-04-25T00:55:56.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbba6ffd640 1 -- 172.21.15.112:0/2611101701 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fbbac016070 con 0x7fbbb00b5800 2024-04-25T00:55:56.403 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:55:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbb940678c0 msgr2=0x7fbb94069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbb940678c0 0x7fbb94069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbbb01386e0 tx=0x7fbba0009040 comp rx=0 tx=0).stop 2024-04-25T00:55:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b5800 msgr2=0x7fbbb00bc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b5800 0x7fbbb00bc410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0ae040 tx=0x7fbbac019100 comp rx=0 tx=0).stop 2024-04-25T00:55:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 shutdown_connections 2024-04-25T00:55:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fbb940678c0 0x7fbb94069d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fbbb01473f0 0x7fbbb01e0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fbbb00b9c60 0x7fbbb0136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 --2- 172.21.15.112:0/2611101701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbbb00b5800 0x7fbbb00bc410 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 >> 172.21.15.112:0/2611101701 conn(0x7fbbb0067120 msgr2=0x7fbbb0136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 shutdown_connections 2024-04-25T00:55:56.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.404+0000 7fbbb8621640 1 -- 172.21.15.112:0/2611101701 wait complete. 2024-04-25T00:55:56.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:55:56.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.540+0000 7f121f21e640 1 Processor -- start 2024-04-25T00:55:56.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.540+0000 7f121f21e640 1 -- start start 2024-04-25T00:55:56.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f121806c170 0x7f12180aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f12180b34b0 0x7f12180b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f121806c550 con 0x7f12180b3e50 2024-04-25T00:55:56.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f12180ae8a0 con 0x7f121806c170 2024-04-25T00:55:56.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f12180ae220 con 0x7f12180b34b0 2024-04-25T00:55:56.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121cfba640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f121806c170 0x7f12180aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121d7bb640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f12180b3e50 0x7f1218147900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50324/0 (socket says 172.21.15.112:50324) 2024-04-25T00:55:56.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121d7bb640 1 -- 172.21.15.112:0/906930272 learned_addr learned my addr 172.21.15.112:0/906930272 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:56.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f120ffff640 1 --2- 172.21.15.112:0/906930272 >> v2:172.21.15.187:3300/0 conn(0x7f12180b34b0 0x7f12180b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121cfba640 1 -- 172.21.15.112:0/906930272 >> v2:172.21.15.187:3300/0 conn(0x7f12180b34b0 msgr2=0x7f12180b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:56.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121cfba640 1 --2- 172.21.15.112:0/906930272 >> v2:172.21.15.187:3300/0 conn(0x7f12180b34b0 0x7f12180b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121cfba640 1 -- 172.21.15.112:0/906930272 >> v1:172.21.15.112:6789/0 conn(0x7f12180b3e50 legacy=0x7f1218147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121cfba640 1 -- 172.21.15.112:0/906930272 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12180b42a0 con 0x7f121806c170 2024-04-25T00:55:56.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f120f7fe640 1 -- 172.21.15.112:0/906930272 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1498533973 0 0) 0x7f121806c550 con 0x7f12180b3e50 2024-04-25T00:55:56.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121cfba640 1 --2- 172.21.15.112:0/906930272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f121806c170 0x7f12180aedd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12180b4b10 tx=0x7f121401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f5a8f9707d4961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f120f7fe640 1 -- 172.21.15.112:0/906930272 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140b0070 con 0x7f121806c170 2024-04-25T00:55:56.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f120f7fe640 1 -- 172.21.15.112:0/906930272 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12140ab600 con 0x7f121806c170 2024-04-25T00:55:56.545 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f120f7fe640 1 -- 172.21.15.112:0/906930272 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12140ab9d0 con 0x7f121806c170 2024-04-25T00:55:56.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 -- 172.21.15.112:0/906930272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f121806c170 msgr2=0x7f12180aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 --2- 172.21.15.112:0/906930272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f121806c170 0x7f12180aedd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f12180b4b10 tx=0x7f121401c3f0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 -- 172.21.15.112:0/906930272 shutdown_connections 2024-04-25T00:55:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 --2- 172.21.15.112:0/906930272 >> v2:172.21.15.187:3300/0 conn(0x7f12180b34b0 0x7f12180b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 --2- 172.21.15.112:0/906930272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f121806c170 0x7f12180aedd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 -- 172.21.15.112:0/906930272 >> 172.21.15.112:0/906930272 conn(0x7f12180b8c80 msgr2=0x7f12180b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 -- 172.21.15.112:0/906930272 shutdown_connections 2024-04-25T00:55:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.544+0000 7f121f21e640 1 -- 172.21.15.112:0/906930272 wait complete. 2024-04-25T00:55:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 Processor -- start 2024-04-25T00:55:56.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 -- start start 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f121806c170 0x7f12181e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12180b34b0 0x7f1218142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12180b3e50 0x7f1218149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f12180b2ea0 con 0x7f12180b34b0 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f12180adef0 con 0x7f12180b3e50 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f12180b51f0 con 0x7f121806c170 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12180b34b0 0x7f1218142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12180b34b0 0x7f1218142510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41900/0 (socket says 172.21.15.112:41900) 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 -- 172.21.15.112:0/1775831333 learned_addr learned my addr 172.21.15.112:0/1775831333 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121cfba640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f121806c170 0x7f12181e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121d7bb640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12180b3e50 0x7f1218149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 -- 172.21.15.112:0/1775831333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f121806c170 msgr2=0x7f12181e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f121806c170 0x7f12181e12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 -- 172.21.15.112:0/1775831333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12180b3e50 msgr2=0x7f1218149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12180b3e50 0x7f1218149d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 -- 172.21.15.112:0/1775831333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121814b560 con 0x7f12180b34b0 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120ffff640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12180b34b0 0x7f1218142510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f120800c820 tx=0x7f120800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120d7fa640 1 -- 172.21.15.112:0/1775831333 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1208015070 con 0x7f12180b34b0 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120d7fa640 1 -- 172.21.15.112:0/1775831333 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1208004240 con 0x7f12180b34b0 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120d7fa640 1 -- 172.21.15.112:0/1775831333 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120800ce10 con 0x7f12180b34b0 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f121814b840 con 0x7f12180b34b0 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1218141d10 con 0x7f12180b34b0 2024-04-25T00:55:56.550 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f121804f5f0 con 0x7f12180b34b0 2024-04-25T00:55:56.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120d7fa640 1 -- 172.21.15.112:0/1775831333 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f12080077d0 con 0x7f12180b34b0 2024-04-25T00:55:56.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.548+0000 7f120d7fa640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f12040678c0 0x7f1204069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.552+0000 7f120d7fa640 1 -- 172.21.15.112:0/1775831333 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1208085a80 con 0x7f12180b34b0 2024-04-25T00:55:56.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.552+0000 7f121cfba640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f12040678c0 0x7f1204069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.552+0000 7f121cfba640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f12040678c0 0x7f1204069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12140ad000 tx=0x7f12140a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.556+0000 7f120d7fa640 1 -- 172.21.15.112:0/1775831333 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1208056650 con 0x7f12180b34b0 2024-04-25T00:55:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.708+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f121806c550 con 0x7f12180b34b0 2024-04-25T00:55:56.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.708+0000 7f120d7fa640 1 -- 172.21.15.112:0/1775831333 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f1208055f50 con 0x7f12180b34b0 2024-04-25T00:55:56.709 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f12040678c0 msgr2=0x7f1204069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f12040678c0 0x7f1204069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12140ad000 tx=0x7f12140a6000 comp rx=0 tx=0).stop 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12180b34b0 msgr2=0x7f1218142510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12180b34b0 0x7f1218142510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f120800c820 tx=0x7f120800cce0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 shutdown_connections 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f12040678c0 0x7f1204069d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f12180b3e50 0x7f1218149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12180b34b0 0x7f1218142510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 --2- 172.21.15.112:0/1775831333 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f121806c170 0x7f12181e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 >> 172.21.15.112:0/1775831333 conn(0x7f12180b8c80 msgr2=0x7f12180b9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:56.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 shutdown_connections 2024-04-25T00:55:56.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.712+0000 7f121f21e640 1 -- 172.21.15.112:0/1775831333 wait complete. 2024-04-25T00:55:56.727 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:56.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.828+0000 7f3d84e86640 1 Processor -- start 2024-04-25T00:55:56.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d84e86640 1 -- start start 2024-04-25T00:55:56.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d84e86640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d800b47d0 0x7f3d800b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d84e86640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d800b5300 con 0x7f3d8006c520 2024-04-25T00:55:56.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d84e86640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8006c900 con 0x7f3d80141110 2024-04-25T00:55:56.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d84e86640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d80072ee0 con 0x7f3d800b47d0 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7e59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3d8006c520 0x7f3d800727d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50328/0 (socket says 172.21.15.112:50328) 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7e59c640 1 -- 172.21.15.112:0/1057555928 learned_addr learned my addr 172.21.15.112:0/1057555928 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1449869079 0 0) 0x7f3d800b5300 con 0x7f3d8006c520 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7dd9b640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d800b47d0 0x7f3d800b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d68003530 con 0x7f3d8006c520 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 449845103 0 0) 0x7f3d68003530 con 0x7f3d8006c520 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d800b47d0 msgr2=0x7f3d800b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d800b47d0 0x7f3d800b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 >> v1:172.21.15.187:6789/0 conn(0x7f3d80141110 legacy=0x7f3d801415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d800ae670 con 0x7f3d8006c520 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f3d74001680 con 0x7f3d8006c520 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 >> v1:172.21.15.112:6789/0 conn(0x7f3d8006c520 legacy=0x7f3d800727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d680039f0 0x7f3d68003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d68004810 0x7f3d68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d800b47d0 0x7f3d68005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80072ee0 con 0x7f3d68004810 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6800bb00 con 0x7f3d680039f0 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6800bd00 con 0x7f3d800b47d0 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7dd9b640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d68004810 0x7f3d68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7e59c640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d680039f0 0x7f3d68003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7dd9b640 1 -- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d800b47d0 msgr2=0x7f3d68005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7dd9b640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d800b47d0 0x7f3d68005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7dd9b640 1 -- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d680039f0 msgr2=0x7f3d68003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7dd9b640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d680039f0 0x7f3d68003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.832+0000 7f3d7dd9b640 1 -- 172.21.15.112:0/1057555928 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d68003530 con 0x7f3d68004810 2024-04-25T00:55:56.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7dd9b640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d68004810 0x7f3d68004c90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009ae0 tx=0x7f3d7001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433b6d3bd8145142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700096b0 con 0x7f3d68004810 2024-04-25T00:55:56.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d700a6490 con 0x7f3d68004810 2024-04-25T00:55:56.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7d59a640 1 -- 172.21.15.112:0/1057555928 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700a6880 con 0x7f3d68004810 2024-04-25T00:55:56.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- 172.21.15.112:0/1057555928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d68004810 msgr2=0x7f3d68004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 --2- 172.21.15.112:0/1057555928 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d68004810 0x7f3d68004c90 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009ae0 tx=0x7f3d7001cab0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 -- 172.21.15.112:0/1057555928 reap_dead start 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- 172.21.15.112:0/1057555928 shutdown_connections 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- 172.21.15.112:0/1057555928 >> 172.21.15.112:0/1057555928 conn(0x7f3d800b5980 msgr2=0x7f3d800b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- 172.21.15.112:0/1057555928 shutdown_connections 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- 172.21.15.112:0/1057555928 wait complete. 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 Processor -- start 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- start start 2024-04-25T00:55:56.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d68004810 0x7f3d8014c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d801503c0 0x7f3d80150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801e3950 0x7f3d801e3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801536b0 con 0x7f3d801e3950 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80153530 con 0x7f3d68004810 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3d80153830 con 0x7f3d801503c0 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801e3950 0x7f3d801e3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801e3950 0x7f3d801e3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41912/0 (socket says 172.21.15.112:41912) 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 -- 172.21.15.112:0/2967022535 learned_addr learned my addr 172.21.15.112:0/2967022535 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7dd9b640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d801503c0 0x7f3d80150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7e59c640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d68004810 0x7f3d8014c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 -- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d801503c0 msgr2=0x7f3d80150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d801503c0 0x7f3d80150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 -- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d68004810 msgr2=0x7f3d8014c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d68004810 0x7f3d8014c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 -- 172.21.15.112:0/2967022535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8014d050 con 0x7f3d801e3950 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7e59c640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d68004810 0x7f3d8014c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7dd9b640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d801503c0 0x7f3d80150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d7ed9d640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801e3950 0x7f3d801e3dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d68004790 tx=0x7f3d7800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d6effd640 1 -- 172.21.15.112:0/2967022535 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78003980 con 0x7f3d801e3950 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d6effd640 1 -- 172.21.15.112:0/2967022535 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d78003b60 con 0x7f3d801e3950 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d6effd640 1 -- 172.21.15.112:0/2967022535 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78004020 con 0x7f3d801e3950 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.836+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d80151690 con 0x7f3d801e3950 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.840+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d80151380 con 0x7f3d801e3950 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.840+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8004f580 con 0x7f3d801e3950 2024-04-25T00:55:56.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.840+0000 7f3d6effd640 1 -- 172.21.15.112:0/2967022535 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3d780041c0 con 0x7f3d801e3950 2024-04-25T00:55:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.840+0000 7f3d6effd640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3d600678c0 0x7f3d60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.840+0000 7f3d6effd640 1 -- 172.21.15.112:0/2967022535 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3d78085040 con 0x7f3d801e3950 2024-04-25T00:55:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.840+0000 7f3d7e59c640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3d600678c0 0x7f3d60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.840+0000 7f3d7e59c640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3d600678c0 0x7f3d60069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3d74000c10 tx=0x7f3d74009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:56.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.844+0000 7f3d6effd640 1 -- 172.21.15.112:0/2967022535 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3d78055c10 con 0x7f3d801e3950 2024-04-25T00:55:56.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.992+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d8014dad0 con 0x7f3d801e3950 2024-04-25T00:55:56.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d6effd640 1 -- 172.21.15.112:0/2967022535 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f3d78055510 con 0x7f3d801e3950 2024-04-25T00:55:56.995 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:56.995 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3d600678c0 msgr2=0x7f3d60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3d600678c0 0x7f3d60069d80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3d74000c10 tx=0x7f3d74009040 comp rx=0 tx=0).stop 2024-04-25T00:55:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801e3950 msgr2=0x7f3d801e3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801e3950 0x7f3d801e3dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d68004790 tx=0x7f3d7800c980 comp rx=0 tx=0).stop 2024-04-25T00:55:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 shutdown_connections 2024-04-25T00:55:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d801e3950 0x7f3d801e3dd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3d801503c0 0x7f3d80150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3d600678c0 0x7f3d60069d80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 --2- 172.21.15.112:0/2967022535 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3d68004810 0x7f3d8014c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 >> 172.21.15.112:0/2967022535 conn(0x7f3d800b5980 msgr2=0x7f3d80147ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 shutdown_connections 2024-04-25T00:55:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:56.996+0000 7f3d84e86640 1 -- 172.21.15.112:0/2967022535 wait complete. 2024-04-25T00:55:57.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:57.013 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:57.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f331c9dd640 1 Processor -- start 2024-04-25T00:55:57.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f331c9dd640 1 -- start start 2024-04-25T00:55:57.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f331c9dd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f33180add40 0x7f33180b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f331c9dd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f331806caf0 0x7f33180b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f331c9dd640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f331809e2b0 con 0x7f331809ded0 2024-04-25T00:55:57.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f331c9dd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f33180ae790 con 0x7f33180add40 2024-04-25T00:55:57.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f331c9dd640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f33180aea70 con 0x7f331806caf0 2024-04-25T00:55:57.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3315d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f33180add40 0x7f33180b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3315d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f33180add40 0x7f33180b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41924/0 (socket says 172.21.15.112:41924) 2024-04-25T00:55:57.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3315d9b640 1 -- 172.21.15.112:0/4063244786 learned_addr learned my addr 172.21.15.112:0/4063244786 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:57.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f331659c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f331809ded0 0x7f33180b34b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50340/0 (socket says 172.21.15.112:50340) 2024-04-25T00:55:57.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3316d9d640 1 --2- 172.21.15.112:0/4063244786 >> v2:172.21.15.187:3300/0 conn(0x7f331806caf0 0x7f33180b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3315d9b640 1 -- 172.21.15.112:0/4063244786 >> v2:172.21.15.187:3300/0 conn(0x7f331806caf0 msgr2=0x7f33180b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:57.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3315d9b640 1 --2- 172.21.15.112:0/4063244786 >> v2:172.21.15.187:3300/0 conn(0x7f331806caf0 0x7f33180b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3315d9b640 1 -- 172.21.15.112:0/4063244786 >> v1:172.21.15.112:6789/0 conn(0x7f331809ded0 legacy=0x7f33180b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3315d9b640 1 -- 172.21.15.112:0/4063244786 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33180aede0 con 0x7f33180add40 2024-04-25T00:55:57.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.124+0000 7f3316d9d640 1 --2- 172.21.15.112:0/4063244786 >> v2:172.21.15.187:3300/0 conn(0x7f331806caf0 0x7f33180b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:57.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f3315d9b640 1 --2- 172.21.15.112:0/4063244786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33180add40 0x7f33180b4980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f331806c7a0 tx=0x7f330c01d860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110eb257f382e896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331559a640 1 -- 172.21.15.112:0/4063244786 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c0b3070 con 0x7f33180add40 2024-04-25T00:55:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331559a640 1 -- 172.21.15.112:0/4063244786 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f330c0a67c0 con 0x7f33180add40 2024-04-25T00:55:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331559a640 1 -- 172.21.15.112:0/4063244786 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c00b040 con 0x7f33180add40 2024-04-25T00:55:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- 172.21.15.112:0/4063244786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33180add40 msgr2=0x7f33180b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 --2- 172.21.15.112:0/4063244786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33180add40 0x7f33180b4980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f331806c7a0 tx=0x7f330c01d860 comp rx=0 tx=0).stop 2024-04-25T00:55:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- 172.21.15.112:0/4063244786 shutdown_connections 2024-04-25T00:55:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 --2- 172.21.15.112:0/4063244786 >> v2:172.21.15.187:3300/0 conn(0x7f331806caf0 0x7f33180b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 --2- 172.21.15.112:0/4063244786 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33180add40 0x7f33180b4980 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- 172.21.15.112:0/4063244786 >> 172.21.15.112:0/4063244786 conn(0x7f33180b8c80 msgr2=0x7f33180b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- 172.21.15.112:0/4063244786 shutdown_connections 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- 172.21.15.112:0/4063244786 wait complete. 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 Processor -- start 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- start start 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331806caf0 0x7f331809be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f331809ded0 0x7f331813a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33180add40 0x7f331813a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f33181504f0 con 0x7f331806caf0 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f33180b4310 con 0x7f331809ded0 2024-04-25T00:55:57.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331c9dd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3318150670 con 0x7f33180add40 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331806caf0 0x7f331809be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331806caf0 0x7f331809be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41928/0 (socket says 172.21.15.112:41928) 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.128+0000 7f331659c640 1 -- 172.21.15.112:0/3469377888 learned_addr learned my addr 172.21.15.112:0/3469377888 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331659c640 1 -- 172.21.15.112:0/3469377888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33180add40 msgr2=0x7f331813a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3316d9d640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33180add40 0x7f331813a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331659c640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33180add40 0x7f331813a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331659c640 1 -- 172.21.15.112:0/3469377888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f331809ded0 msgr2=0x7f331813a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331659c640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f331809ded0 0x7f331813a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331659c640 1 -- 172.21.15.112:0/3469377888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331809d860 con 0x7f331806caf0 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331659c640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331806caf0 0x7f331809be60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f330800c420 tx=0x7f330800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.134 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3306ffd640 1 -- 172.21.15.112:0/3469377888 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33080074e0 con 0x7f331806caf0 2024-04-25T00:55:57.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3306ffd640 1 -- 172.21.15.112:0/3469377888 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3308007680 con 0x7f331806caf0 2024-04-25T00:55:57.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3306ffd640 1 -- 172.21.15.112:0/3469377888 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3308004a30 con 0x7f331806caf0 2024-04-25T00:55:57.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331809b000 con 0x7f331806caf0 2024-04-25T00:55:57.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331809b5c0 con 0x7f331806caf0 2024-04-25T00:55:57.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f331806d800 con 0x7f331806caf0 2024-04-25T00:55:57.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3306ffd640 1 -- 172.21.15.112:0/3469377888 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f330801a020 con 0x7f331806caf0 2024-04-25T00:55:57.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3306ffd640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f32f80678c0 0x7f32f8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3306ffd640 1 -- 172.21.15.112:0/3469377888 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3308085440 con 0x7f331806caf0 2024-04-25T00:55:57.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3315d9b640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f32f80678c0 0x7f32f8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.132+0000 7f3315d9b640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f32f80678c0 0x7f32f8069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f330c0b0000 tx=0x7f330c00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.136 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.136+0000 7f3306ffd640 1 -- 172.21.15.112:0/3469377888 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3308056010 con 0x7f331806caf0 2024-04-25T00:55:57.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.292+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f331804f5f0 con 0x7f331806caf0 2024-04-25T00:55:57.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f3306ffd640 1 -- 172.21.15.112:0/3469377888 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f3308055910 con 0x7f331806caf0 2024-04-25T00:55:57.294 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f32f80678c0 msgr2=0x7f32f8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f32f80678c0 0x7f32f8069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f330c0b0000 tx=0x7f330c00c040 comp rx=0 tx=0).stop 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331806caf0 msgr2=0x7f331809be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331806caf0 0x7f331809be60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f330800c420 tx=0x7f330800c8e0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 shutdown_connections 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f33180add40 0x7f331813a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f32f80678c0 0x7f32f8069d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f331809ded0 0x7f331813a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 --2- 172.21.15.112:0/3469377888 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f331806caf0 0x7f331809be60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 >> 172.21.15.112:0/3469377888 conn(0x7f33180b8c80 msgr2=0x7f3318143a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:57.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 shutdown_connections 2024-04-25T00:55:57.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.296+0000 7f331c9dd640 1 -- 172.21.15.112:0/3469377888 wait complete. 2024-04-25T00:55:57.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:55:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e255a640 1 Processor -- start 2024-04-25T00:55:57.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e255a640 1 -- start start 2024-04-25T00:55:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e255a640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f40dc0adec0 0x7f40dc14ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e255a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40dc14b6f0 0x7f40dc151f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e255a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40dc0affb0 con 0x7f40dc14b240 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e255a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f40dc06d630 con 0x7f40dc14b6f0 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e255a640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f40dc14bb60 con 0x7f40dc0adec0 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e0af7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40dc14b6f0 0x7f40dc151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e0af7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f40dc14b6f0 0x7f40dc151f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41934/0 (socket says 172.21.15.112:41934) 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e0af7640 1 -- 172.21.15.112:0/1909166913 learned_addr learned my addr 172.21.15.112:0/1909166913 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40dbfff640 1 --2- 172.21.15.112:0/1909166913 >> v2:172.21.15.187:3300/0 conn(0x7f40dc0adec0 0x7f40dc14ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e0af7640 1 -- 172.21.15.112:0/1909166913 >> v2:172.21.15.187:3300/0 conn(0x7f40dc0adec0 msgr2=0x7f40dc14ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e0af7640 1 --2- 172.21.15.112:0/1909166913 >> v2:172.21.15.187:3300/0 conn(0x7f40dc0adec0 0x7f40dc14ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e0af7640 1 -- 172.21.15.112:0/1909166913 >> v1:172.21.15.187:6789/0 conn(0x7f40dc14b240 legacy=0x7f40dc06bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40e0af7640 1 -- 172.21.15.112:0/1909166913 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc0af2f0 con 0x7f40dc14b6f0 2024-04-25T00:55:57.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.432+0000 7f40dbfff640 1 --2- 172.21.15.112:0/1909166913 >> v2:172.21.15.187:3300/0 conn(0x7f40dc0adec0 0x7f40dc14ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e0af7640 1 --2- 172.21.15.112:0/1909166913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc14b6f0 0x7f40dc151f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0b3e30 tx=0x7f40d401c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fad46a213607f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40daffd640 1 -- 172.21.15.112:0/1909166913 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d40ac020 con 0x7f40dc14b6f0 2024-04-25T00:55:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40daffd640 1 -- 172.21.15.112:0/1909166913 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d40a64a0 con 0x7f40dc14b6f0 2024-04-25T00:55:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40daffd640 1 -- 172.21.15.112:0/1909166913 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d40a6870 con 0x7f40dc14b6f0 2024-04-25T00:55:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- 172.21.15.112:0/1909166913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc14b6f0 msgr2=0x7f40dc151f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 --2- 172.21.15.112:0/1909166913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc14b6f0 0x7f40dc151f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0b3e30 tx=0x7f40d401c7d0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- 172.21.15.112:0/1909166913 shutdown_connections 2024-04-25T00:55:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 --2- 172.21.15.112:0/1909166913 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc14b6f0 0x7f40dc151f50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.435 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 --2- 172.21.15.112:0/1909166913 >> v2:172.21.15.187:3300/0 conn(0x7f40dc0adec0 0x7f40dc14ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- 172.21.15.112:0/1909166913 >> 172.21.15.112:0/1909166913 conn(0x7f40dc0b8c80 msgr2=0x7f40dc0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:57.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- 172.21.15.112:0/1909166913 shutdown_connections 2024-04-25T00:55:57.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- 172.21.15.112:0/1909166913 wait complete. 2024-04-25T00:55:57.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 Processor -- start 2024-04-25T00:55:57.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- start start 2024-04-25T00:55:57.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0adec0 0x7f40dc146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40dc14b240 0x7f40dc09d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40dc14b6f0 0x7f40dc09d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc0afa60 con 0x7f40dc0adec0 2024-04-25T00:55:57.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc0ae2a0 con 0x7f40dc14b240 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f40dc0afbe0 con 0x7f40dc14b6f0 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0adec0 0x7f40dc146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0adec0 0x7f40dc146cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41944/0 (socket says 172.21.15.112:41944) 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 -- 172.21.15.112:0/1720404554 learned_addr learned my addr 172.21.15.112:0/1720404554 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e0af7640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40dc14b6f0 0x7f40dc09d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40db7fe640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40dc14b240 0x7f40dc09d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 -- 172.21.15.112:0/1720404554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40dc14b6f0 msgr2=0x7f40dc09d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40dc14b6f0 0x7f40dc09d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 -- 172.21.15.112:0/1720404554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40dc14b240 msgr2=0x7f40dc09d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40dc14b240 0x7f40dc09d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 -- 172.21.15.112:0/1720404554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc14de50 con 0x7f40dc0adec0 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40dbfff640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0adec0 0x7f40dc146cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00c7c0 tx=0x7f40cc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40d8ff9640 1 -- 172.21.15.112:0/1720404554 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc007a30 con 0x7f40dc0adec0 2024-04-25T00:55:57.439 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40d8ff9640 1 -- 172.21.15.112:0/1720404554 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40cc007bd0 con 0x7f40dc0adec0 2024-04-25T00:55:57.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc09c690 con 0x7f40dc0adec0 2024-04-25T00:55:57.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40d8ff9640 1 -- 172.21.15.112:0/1720404554 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0113f0 con 0x7f40dc0adec0 2024-04-25T00:55:57.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.436+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc14cbc0 con 0x7f40dc0adec0 2024-04-25T00:55:57.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.440+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc14cfe0 con 0x7f40dc0adec0 2024-04-25T00:55:57.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.440+0000 7f40d8ff9640 1 -- 172.21.15.112:0/1720404554 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40cc011590 con 0x7f40dc0adec0 2024-04-25T00:55:57.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.440+0000 7f40d8ff9640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f40c80678c0 0x7f40c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.440+0000 7f40d8ff9640 1 -- 172.21.15.112:0/1720404554 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f40cc085dc0 con 0x7f40dc0adec0 2024-04-25T00:55:57.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.440+0000 7f40db7fe640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f40c80678c0 0x7f40c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.440+0000 7f40db7fe640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f40c80678c0 0x7f40c8069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f40dc14c2c0 tx=0x7f40d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.444+0000 7f40d8ff9640 1 -- 172.21.15.112:0/1720404554 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f40cc015020 con 0x7f40dc0adec0 2024-04-25T00:55:57.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.604+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f40dc147b20 con 0x7f40dc0adec0 2024-04-25T00:55:57.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.604+0000 7f40d8ff9640 1 -- 172.21.15.112:0/1720404554 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f40cc056990 con 0x7f40dc0adec0 2024-04-25T00:55:57.605 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:55:57.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f40c80678c0 msgr2=0x7f40c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f40c80678c0 0x7f40c8069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f40dc14c2c0 tx=0x7f40d0009040 comp rx=0 tx=0).stop 2024-04-25T00:55:57.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0adec0 msgr2=0x7f40dc146cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0adec0 0x7f40dc146cc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f40cc00c7c0 tx=0x7f40cc00cc80 comp rx=0 tx=0).stop 2024-04-25T00:55:57.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 shutdown_connections 2024-04-25T00:55:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f40dc14b6f0 0x7f40dc09d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f40c80678c0 0x7f40c8069d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f40dc14b240 0x7f40dc09d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 --2- 172.21.15.112:0/1720404554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f40dc0adec0 0x7f40dc146cc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 >> 172.21.15.112:0/1720404554 conn(0x7f40dc0b8c80 msgr2=0x7f40dc0b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 shutdown_connections 2024-04-25T00:55:57.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.608+0000 7f40e255a640 1 -- 172.21.15.112:0/1720404554 wait complete. 2024-04-25T00:55:57.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:55:57.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.748+0000 7f4a9276b640 1 Processor -- start 2024-04-25T00:55:57.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a9276b640 1 -- start start 2024-04-25T00:55:57.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a9276b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c06c170 0x7f4a8c0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a9276b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f4a8c0b34b0 0x7f4a8c0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a9276b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a8c0b3e50 0x7f4a8c147900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a9276b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c0ae8a0 con 0x7f4a8c0b3e50 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a9276b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c0b42e0 con 0x7f4a8c0b34b0 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a9276b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c06c550 con 0x7f4a8c06c170 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a8c0b3e50 0x7f4a8c147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a8c0b3e50 0x7f4a8c147900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41960/0 (socket says 172.21.15.112:41960) 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 -- 172.21.15.112:0/120134779 learned_addr learned my addr 172.21.15.112:0/120134779 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a8bfff640 1 --2- 172.21.15.112:0/120134779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c06c170 0x7f4a8c0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a8b7fe640 1 --2- 172.21.15.112:0/120134779 >> v2:172.21.15.187:3300/0 conn(0x7f4a8c0b34b0 0x7f4a8c0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 -- 172.21.15.112:0/120134779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c06c170 msgr2=0x7f4a8c0aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 --2- 172.21.15.112:0/120134779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c06c170 0x7f4a8c0aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 -- 172.21.15.112:0/120134779 >> v2:172.21.15.187:3300/0 conn(0x7f4a8c0b34b0 msgr2=0x7f4a8c0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 --2- 172.21.15.112:0/120134779 >> v2:172.21.15.187:3300/0 conn(0x7f4a8c0b34b0 0x7f4a8c0b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 -- 172.21.15.112:0/120134779 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c0ae220 con 0x7f4a8c0b3e50 2024-04-25T00:55:57.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a90d08640 1 --2- 172.21.15.112:0/120134779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c0b3e50 0x7f4a8c147900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009870 tx=0x7f4a8401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f73e0de83be2ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a8affd640 1 -- 172.21.15.112:0/120134779 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a840b0070 con 0x7f4a8c0b3e50 2024-04-25T00:55:57.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a8affd640 1 -- 172.21.15.112:0/120134779 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a840ab600 con 0x7f4a8c0b3e50 2024-04-25T00:55:57.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.752+0000 7f4a8affd640 1 -- 172.21.15.112:0/120134779 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a840ab9d0 con 0x7f4a8c0b3e50 2024-04-25T00:55:57.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- 172.21.15.112:0/120134779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c0b3e50 msgr2=0x7f4a8c147900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 --2- 172.21.15.112:0/120134779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c0b3e50 0x7f4a8c147900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4a84009870 tx=0x7f4a8401c1d0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- 172.21.15.112:0/120134779 shutdown_connections 2024-04-25T00:55:57.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 --2- 172.21.15.112:0/120134779 >> v2:172.21.15.187:3300/0 conn(0x7f4a8c0b34b0 0x7f4a8c0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 --2- 172.21.15.112:0/120134779 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c0b3e50 0x7f4a8c147900 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 --2- 172.21.15.112:0/120134779 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c06c170 0x7f4a8c0aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- 172.21.15.112:0/120134779 >> 172.21.15.112:0/120134779 conn(0x7f4a8c0b8c80 msgr2=0x7f4a8c0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:57.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- 172.21.15.112:0/120134779 shutdown_connections 2024-04-25T00:55:57.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- 172.21.15.112:0/120134779 wait complete. 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 Processor -- start 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- start start 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c06c170 0x7f4a8c146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c0b34b0 0x7f4a8c1436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c0b3e50 0x7f4a8c143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0b2e80 con 0x7f4a8c06c170 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0adef0 con 0x7f4a8c0b34b0 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c0b51f0 con 0x7f4a8c0b3e50 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c06c170 0x7f4a8c146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c06c170 0x7f4a8c146bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41964/0 (socket says 172.21.15.112:41964) 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 -- 172.21.15.112:0/3516544825 learned_addr learned my addr 172.21.15.112:0/3516544825 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:57.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8b7fe640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c0b34b0 0x7f4a8c1436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a90d08640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c0b3e50 0x7f4a8c143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 -- 172.21.15.112:0/3516544825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c0b3e50 msgr2=0x7f4a8c143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c0b3e50 0x7f4a8c143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 -- 172.21.15.112:0/3516544825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c0b34b0 msgr2=0x7f4a8c1436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c0b34b0 0x7f4a8c1436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 -- 172.21.15.112:0/3516544825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c071790 con 0x7f4a8c06c170 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a8bfff640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c06c170 0x7f4a8c146bc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c007e40 tx=0x7f4a7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a88ff9640 1 -- 172.21.15.112:0/3516544825 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c014620 con 0x7f4a8c06c170 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a88ff9640 1 -- 172.21.15.112:0/3516544825 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c019070 con 0x7f4a8c06c170 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c142c90 con 0x7f4a8c06c170 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a88ff9640 1 -- 172.21.15.112:0/3516544825 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c014a00 con 0x7f4a8c06c170 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c142070 con 0x7f4a8c06c170 2024-04-25T00:55:57.759 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.756+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8c0b9420 con 0x7f4a8c06c170 2024-04-25T00:55:57.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.760+0000 7f4a88ff9640 1 -- 172.21.15.112:0/3516544825 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4a7c014ba0 con 0x7f4a8c06c170 2024-04-25T00:55:57.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.760+0000 7f4a88ff9640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4a6c0678c0 0x7f4a6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:57.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.760+0000 7f4a88ff9640 1 -- 172.21.15.112:0/3516544825 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a7c086580 con 0x7f4a8c06c170 2024-04-25T00:55:57.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.760+0000 7f4a8b7fe640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4a6c0678c0 0x7f4a6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:57.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.760+0000 7f4a8b7fe640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4a6c0678c0 0x7f4a6c069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c13cd70 tx=0x7f4a74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:57.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.764+0000 7f4a88ff9640 1 -- 172.21.15.112:0/3516544825 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4a7c0570b0 con 0x7f4a8c06c170 2024-04-25T00:55:57.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.920+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a8c04f5f0 con 0x7f4a8c06c170 2024-04-25T00:55:57.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a88ff9640 1 -- 172.21.15.112:0/3516544825 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4a7c0569b0 con 0x7f4a8c06c170 2024-04-25T00:55:57.922 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:55:57.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4a6c0678c0 msgr2=0x7f4a6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4a6c0678c0 0x7f4a6c069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c13cd70 tx=0x7f4a74008040 comp rx=0 tx=0).stop 2024-04-25T00:55:57.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c06c170 msgr2=0x7f4a8c146bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:57.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c06c170 0x7f4a8c146bc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c007e40 tx=0x7f4a7c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 shutdown_connections 2024-04-25T00:55:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4a8c0b3e50 0x7f4a8c143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4a6c0678c0 0x7f4a6c069d80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4a8c0b34b0 0x7f4a8c1436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 --2- 172.21.15.112:0/3516544825 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a8c06c170 0x7f4a8c146bc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 >> 172.21.15.112:0/3516544825 conn(0x7f4a8c0b8c80 msgr2=0x7f4a8c0b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 shutdown_connections 2024-04-25T00:55:57.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:57.924+0000 7f4a9276b640 1 -- 172.21.15.112:0/3516544825 wait complete. 2024-04-25T00:55:57.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:58.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d8d13640 1 Processor -- start 2024-04-25T00:55:58.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d8d13640 1 -- start start 2024-04-25T00:55:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d8d13640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 0x7fe4d40add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d8d13640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4d4145200 0x7fe4d40b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d8d13640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d40aeec0 con 0x7fe4d4072760 2024-04-25T00:55:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d8d13640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d40af780 con 0x7fe4d4145200 2024-04-25T00:55:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d8d13640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d40b27d0 con 0x7fe4d40b23f0 2024-04-25T00:55:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4cbfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4d4145200 0x7fe4d40b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d2d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe4d4072760 0x7fe4d406c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50354/0 (socket says 172.21.15.112:50354) 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4cbfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4d4145200 0x7fe4d40b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41968/0 (socket says 172.21.15.112:41968) 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d2d9d640 1 -- 172.21.15.112:0/4071669601 learned_addr learned my addr 172.21.15.112:0/4071669601 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d259c640 1 --2- 172.21.15.112:0/4071669601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 0x7fe4d40add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4cbfff640 1 -- 172.21.15.112:0/4071669601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 msgr2=0x7fe4d40add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4cbfff640 1 --2- 172.21.15.112:0/4071669601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 0x7fe4d40add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4cbfff640 1 -- 172.21.15.112:0/4071669601 >> v1:172.21.15.112:6789/0 conn(0x7fe4d4072760 legacy=0x7fe4d406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4cbfff640 1 -- 172.21.15.112:0/4071669601 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d40afa20 con 0x7fe4d4145200 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.060+0000 7fe4d1d9b640 1 -- 172.21.15.112:0/4071669601 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 503675385 0 0) 0x7fe4d40aeec0 con 0x7fe4d4072760 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4cbfff640 1 --2- 172.21.15.112:0/4071669601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d40b3ce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40097c0 tx=0x7fe4c401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c0641074d36d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d1d9b640 1 -- 172.21.15.112:0/4071669601 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c40aa620 con 0x7fe4d4145200 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d1d9b640 1 -- 172.21.15.112:0/4071669601 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c40af070 con 0x7fe4d4145200 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d1d9b640 1 -- 172.21.15.112:0/4071669601 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c40aa9e0 con 0x7fe4d4145200 2024-04-25T00:55:58.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- 172.21.15.112:0/4071669601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 msgr2=0x7fe4d40b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/4071669601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d40b3ce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4c40097c0 tx=0x7fe4c401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- 172.21.15.112:0/4071669601 shutdown_connections 2024-04-25T00:55:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/4071669601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d40b3ce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/4071669601 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 0x7fe4d40add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- 172.21.15.112:0/4071669601 >> 172.21.15.112:0/4071669601 conn(0x7fe4d40b5980 msgr2=0x7fe4d40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- 172.21.15.112:0/4071669601 shutdown_connections 2024-04-25T00:55:58.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- 172.21.15.112:0/4071669601 wait complete. 2024-04-25T00:55:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 Processor -- start 2024-04-25T00:55:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- start start 2024-04-25T00:55:58.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d4072760 0x7fe4d414f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 0x7fe4d41e0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d41e0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40b5360 con 0x7fe4d4145200 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4072fa0 con 0x7fe4d4072760 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d8d13640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d4152d70 con 0x7fe4d40b23f0 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d41e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d2d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d41e0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:41984/0 (socket says 172.21.15.112:41984) 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d2d9d640 1 -- 172.21.15.112:0/3630288481 learned_addr learned my addr 172.21.15.112:0/3630288481 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4cbfff640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 0x7fe4d41e0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d2d9d640 1 -- 172.21.15.112:0/3630288481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 msgr2=0x7fe4d41e0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d2d9d640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 0x7fe4d41e0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d2d9d640 1 -- 172.21.15.112:0/3630288481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d4072760 msgr2=0x7fe4d414f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d2d9d640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d4072760 0x7fe4d414f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.064+0000 7fe4d2d9d640 1 -- 172.21.15.112:0/3630288481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d4139310 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4d2d9d640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d41e0990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c7b0 tx=0x7fe4cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4caffd640 1 -- 172.21.15.112:0/3630288481 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc018070 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4caffd640 1 -- 172.21.15.112:0/3630288481 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc00a640 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d4150080 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4caffd640 1 -- 172.21.15.112:0/3630288481 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc00ce60 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d414fd80 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d404f580 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4caffd640 1 -- 172.21.15.112:0/3630288481 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe4cc01b070 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4caffd640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4b40678c0 0x7fe4b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4caffd640 1 -- 172.21.15.112:0/3630288481 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe4cc085650 con 0x7fe4d4145200 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4d259c640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4b40678c0 0x7fe4b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.068+0000 7fe4d259c640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4b40678c0 0x7fe4b4069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0098f0 tx=0x7fe4bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.072 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.072+0000 7fe4caffd640 1 -- 172.21.15.112:0/3630288481 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe4cc022020 con 0x7fe4d4145200 2024-04-25T00:55:58.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.220+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4d40b8880 con 0x7fe4d4145200 2024-04-25T00:55:58.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4caffd640 1 -- 172.21.15.112:0/3630288481 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fe4cc056300 con 0x7fe4d4145200 2024-04-25T00:55:58.224 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:58.224 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:58.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4b40678c0 msgr2=0x7fe4b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4b40678c0 0x7fe4b4069d80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc0098f0 tx=0x7fe4bc008040 comp rx=0 tx=0).stop 2024-04-25T00:55:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 msgr2=0x7fe4d41e0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d41e0990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c7b0 tx=0x7fe4cc00cc70 comp rx=0 tx=0).stop 2024-04-25T00:55:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 shutdown_connections 2024-04-25T00:55:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4d4145200 0x7fe4d41e0990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4d40b23f0 0x7fe4d41e0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4b40678c0 0x7fe4b4069d80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 --2- 172.21.15.112:0/3630288481 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4d4072760 0x7fe4d414f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 >> 172.21.15.112:0/3630288481 conn(0x7fe4d40b5980 msgr2=0x7fe4d40b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 shutdown_connections 2024-04-25T00:55:58.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.224+0000 7fe4d8d13640 1 -- 172.21.15.112:0/3630288481 wait complete. 2024-04-25T00:55:58.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:58.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:58.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0c02032640 1 Processor -- start 2024-04-25T00:55:58.355 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0c02032640 1 -- start start 2024-04-25T00:55:58.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0c02032640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc09b800 0x7f0bfc141350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0c02032640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bfc0b9410 con 0x7f0bfc0b9030 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0c02032640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bfc0baa90 con 0x7f0bfc141890 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0c02032640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc09b3c0 con 0x7f0bfc09b800 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfb7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f0bfc0b9030 0x7f0bfc09acb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50366/0 (socket says 172.21.15.112:50366) 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfb7fe640 1 -- 172.21.15.112:0/1492257756 learned_addr learned my addr 172.21.15.112:0/1492257756 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfaffd640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc09b800 0x7f0bfc141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197045067 0 0) 0x7f0bfc0b9410 con 0x7f0bfc0b9030 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0be4003530 con 0x7f0bfc0b9030 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1658252783 0 0) 0x7f0be4003530 con 0x7f0bfc0b9030 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc09b800 msgr2=0x7f0bfc141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.358 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc09b800 0x7f0bfc141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 >> v1:172.21.15.187:6789/0 conn(0x7f0bfc141890 legacy=0x7f0bfc151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc0bad70 con 0x7f0bfc0b9030 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f0bf0001680 con 0x7f0bfc0b9030 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 >> v1:172.21.15.112:6789/0 conn(0x7f0bfc0b9030 legacy=0x7f0bfc09acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc09b800 0x7f0be40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be4003f70 0x7f0be4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be4004d50 0x7f0be40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc09b3c0 con 0x7f0be4004d50 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0be400bb60 con 0x7f0be4003f70 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0be400bd60 con 0x7f0bfc09b800 2024-04-25T00:55:58.359 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfbfff640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be4004d50 0x7f0be40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfb7fe640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc09b800 0x7f0be40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfaffd640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be4003f70 0x7f0be4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfbfff640 1 -- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc09b800 msgr2=0x7f0be40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfbfff640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc09b800 0x7f0be40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfbfff640 1 -- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be4003f70 msgr2=0x7f0be4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfbfff640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0be4003f70 0x7f0be4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.360 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.356+0000 7f0bfbfff640 1 -- 172.21.15.112:0/1492257756 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be4003530 con 0x7f0be4004d50 2024-04-25T00:55:58.363 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfbfff640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be4004d50 0x7f0be40051d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4001e40 tx=0x7f0bf401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166760e1e075fff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4001a90 con 0x7f0be4004d50 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf4003b80 con 0x7f0be4004d50 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfa7fc640 1 -- 172.21.15.112:0/1492257756 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4005080 con 0x7f0be4004d50 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- 172.21.15.112:0/1492257756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be4004d50 msgr2=0x7f0be40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 --2- 172.21.15.112:0/1492257756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0be4004d50 0x7f0be40051d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4001e40 tx=0x7f0bf401c760 comp rx=0 tx=0).stop 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfbfff640 1 -- 172.21.15.112:0/1492257756 reap_dead start 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- 172.21.15.112:0/1492257756 shutdown_connections 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- 172.21.15.112:0/1492257756 >> 172.21.15.112:0/1492257756 conn(0x7f0bfc09eaf0 msgr2=0x7f0bfc09ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- 172.21.15.112:0/1492257756 shutdown_connections 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- 172.21.15.112:0/1492257756 wait complete. 2024-04-25T00:55:58.364 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 Processor -- start 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- start start 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b6400 0x7f0bfc1450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bfc143ba0 0x7f0bfc144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc141d70 0x7f0bfc1421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc09bc70 con 0x7f0bfc0b6400 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0bfc0b9990 con 0x7f0bfc143ba0 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0c02032640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0bfc0b9e20 con 0x7f0bfc141d70 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b6400 0x7f0bfc1450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfb7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b6400 0x7f0bfc1450a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42008/0 (socket says 172.21.15.112:42008) 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfb7fe640 1 -- 172.21.15.112:0/4170290339 learned_addr learned my addr 172.21.15.112:0/4170290339 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfbfff640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc141d70 0x7f0bfc1421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.360+0000 7f0bfaffd640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bfc143ba0 0x7f0bfc144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.365 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0bfb7fe640 1 -- 172.21.15.112:0/4170290339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc141d70 msgr2=0x7f0bfc1421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0bfb7fe640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc141d70 0x7f0bfc1421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0bfb7fe640 1 -- 172.21.15.112:0/4170290339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bfc143ba0 msgr2=0x7f0bfc144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0bfb7fe640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bfc143ba0 0x7f0bfc144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0bfb7fe640 1 -- 172.21.15.112:0/4170290339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bfc1484e0 con 0x7f0bfc0b6400 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0bfb7fe640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b6400 0x7f0bfc1450a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0011980 tx=0x7f0bf0011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0be3fff640 1 -- 172.21.15.112:0/4170290339 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0010040 con 0x7f0bfc0b6400 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0be3fff640 1 -- 172.21.15.112:0/4170290339 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf001a070 con 0x7f0bfc0b6400 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bfc0b67e0 con 0x7f0bfc0b6400 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0be3fff640 1 -- 172.21.15.112:0/4170290339 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0015410 con 0x7f0bfc0b6400 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bfc144bc0 con 0x7f0bfc0b6400 2024-04-25T00:55:58.366 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bfc0b6fe0 con 0x7f0bfc0b6400 2024-04-25T00:55:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0be3fff640 1 -- 172.21.15.112:0/4170290339 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0bf00155b0 con 0x7f0bfc0b6400 2024-04-25T00:55:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0be3fff640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0be80678f0 0x7f0be8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0be3fff640 1 -- 172.21.15.112:0/4170290339 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0bf00899d0 con 0x7f0bfc0b6400 2024-04-25T00:55:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0bfaffd640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0be80678f0 0x7f0be8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.367 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.364+0000 7f0bfaffd640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0be80678f0 0x7f0be8069db0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0b6c70 tx=0x7f0bec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.368 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.368+0000 7f0be3fff640 1 -- 172.21.15.112:0/4170290339 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0bf005a680 con 0x7f0bfc0b6400 2024-04-25T00:55:58.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.524+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0bfc04f5f0 con 0x7f0bfc0b6400 2024-04-25T00:55:58.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.524+0000 7f0be3fff640 1 -- 172.21.15.112:0/4170290339 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0bf0059f80 con 0x7f0bfc0b6400 2024-04-25T00:55:58.524 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:58.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.524+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0be80678f0 msgr2=0x7f0be8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.524+0000 7f0c02032640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0be80678f0 0x7f0be8069db0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc0b6c70 tx=0x7f0bec008040 comp rx=0 tx=0).stop 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.524+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b6400 msgr2=0x7f0bfc1450a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.524+0000 7f0c02032640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b6400 0x7f0bfc1450a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0011980 tx=0x7f0bf0011e40 comp rx=0 tx=0).stop 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.528+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 shutdown_connections 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.528+0000 7f0c02032640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0bfc141d70 0x7f0bfc1421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.528+0000 7f0c02032640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0be80678f0 0x7f0be8069db0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.528+0000 7f0c02032640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0bfc143ba0 0x7f0bfc144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.528+0000 7f0c02032640 1 --2- 172.21.15.112:0/4170290339 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0bfc0b6400 0x7f0bfc1450a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.528+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 >> 172.21.15.112:0/4170290339 conn(0x7f0bfc09eaf0 msgr2=0x7f0bfc069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.528+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 shutdown_connections 2024-04-25T00:55:58.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.528+0000 7f0c02032640 1 -- 172.21.15.112:0/4170290339 wait complete. 2024-04-25T00:55:58.543 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:55:58.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57bd73d640 1 Processor -- start 2024-04-25T00:55:58.663 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57bd73d640 1 -- start start 2024-04-25T00:55:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57bd73d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f57b8147900 0x7f57b8147d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57bd73d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b80af4e0 con 0x7f57b806c170 2024-04-25T00:55:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57bd73d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b806c550 con 0x7f57b81482a0 2024-04-25T00:55:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57bd73d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f57b80ae8a0 con 0x7f57b8147900 2024-04-25T00:55:58.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b6ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f57b806c170 0x7f57b80aedd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50380/0 (socket says 172.21.15.112:50380) 2024-04-25T00:55:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b6ffd640 1 -- 172.21.15.112:0/588007499 learned_addr learned my addr 172.21.15.112:0/588007499 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57ae7fc640 1 --2- 172.21.15.112:0/588007499 >> v2:172.21.15.187:3300/0 conn(0x7f57b8147900 0x7f57b8147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913050787 0 0) 0x7f57b80af4e0 con 0x7f57b806c170 2024-04-25T00:55:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57a4005740 con 0x7f57b806c170 2024-04-25T00:55:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 875677699 0 0) 0x7f57a4005740 con 0x7f57b806c170 2024-04-25T00:55:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 >> v2:172.21.15.187:3300/0 conn(0x7f57b8147900 msgr2=0x7f57b8147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 --2- 172.21.15.112:0/588007499 >> v2:172.21.15.187:3300/0 conn(0x7f57b8147900 0x7f57b8147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 >> v1:172.21.15.187:6789/0 conn(0x7f57b81482a0 legacy=0x7f57b814ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b80ae220 con 0x7f57b806c170 2024-04-25T00:55:58.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f57a8001680 con 0x7f57b806c170 2024-04-25T00:55:58.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 >> v1:172.21.15.112:6789/0 conn(0x7f57b806c170 legacy=0x7f57b80aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57a4005bd0 0x7f57a4005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a4006a20 0x7f57a4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57a40073e0 0x7f57a400ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4010550 con 0x7f57a4006a20 2024-04-25T00:55:58.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4010700 con 0x7f57a4005bd0 2024-04-25T00:55:58.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57a4010900 con 0x7f57a40073e0 2024-04-25T00:55:58.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57ae7fc640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a4006a20 0x7f57a4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b6ffd640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57a4005bd0 0x7f57a4005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57ae7fc640 1 -- 172.21.15.112:0/588007499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57a40073e0 msgr2=0x7f57a400ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57ae7fc640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57a40073e0 0x7f57a400ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57ae7fc640 1 -- 172.21.15.112:0/588007499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57a4005bd0 msgr2=0x7f57a4005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57ae7fc640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57a4005bd0 0x7f57a4005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57ae7fc640 1 -- 172.21.15.112:0/588007499 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a4005740 con 0x7f57a4006a20 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b6ffd640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57a4005bd0 0x7f57a4005ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.664+0000 7f57b77fe640 1 -- 172.21.15.112:0/588007499 reap_dead start 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57ae7fc640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a4006a20 0x7f57a4006ea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009ae0 tx=0x7f57a001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c63bce5deb5de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a00096b0 con 0x7f57a4006a20 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a00a6470 con 0x7f57a4006a20 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b67fc640 1 -- 172.21.15.112:0/588007499 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a00a6860 con 0x7f57a4006a20 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- 172.21.15.112:0/588007499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a4006a20 msgr2=0x7f57a4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a4006a20 0x7f57a4006ea0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009ae0 tx=0x7f57a001ca90 comp rx=0 tx=0).stop 2024-04-25T00:55:58.675 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- 172.21.15.112:0/588007499 shutdown_connections 2024-04-25T00:55:58.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 --2- 172.21.15.112:0/588007499 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57a4006a20 0x7f57a4006ea0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- 172.21.15.112:0/588007499 >> 172.21.15.112:0/588007499 conn(0x7f57b80b8c80 msgr2=0x7f57b80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:58.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- 172.21.15.112:0/588007499 shutdown_connections 2024-04-25T00:55:58.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- 172.21.15.112:0/588007499 wait complete. 2024-04-25T00:55:58.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 Processor -- start 2024-04-25T00:55:58.676 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- start start 2024-04-25T00:55:58.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57a4006a20 0x7f57b809d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b809aaa0 0x7f57b81dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b8070420 0x7f57b80708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f57b80adef0 con 0x7f57b8070420 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f57b80ae530 con 0x7f57b809aaa0 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f57b80b2e80 con 0x7f57a4006a20 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b8070420 0x7f57b80708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b8070420 0x7f57b80708d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42028/0 (socket says 172.21.15.112:42028) 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 -- 172.21.15.112:0/4202548335 learned_addr learned my addr 172.21.15.112:0/4202548335 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b6ffd640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57a4006a20 0x7f57b809d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 -- 172.21.15.112:0/4202548335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57a4006a20 msgr2=0x7f57b809d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57a4006a20 0x7f57b809d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57ae7fc640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b809aaa0 0x7f57b81dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 -- 172.21.15.112:0/4202548335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b809aaa0 msgr2=0x7f57b81dbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b809aaa0 0x7f57b81dbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.679 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 -- 172.21.15.112:0/4202548335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b8070e10 con 0x7f57b8070420 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57b77fe640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b8070420 0x7f57b80708d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57b80b5160 tx=0x7f57b000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57affff640 1 -- 172.21.15.112:0/4202548335 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0003980 con 0x7f57b8070420 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57affff640 1 -- 172.21.15.112:0/4202548335 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57b0003b40 con 0x7f57b8070420 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b8071720 con 0x7f57b8070420 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57affff640 1 -- 172.21.15.112:0/4202548335 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b0004020 con 0x7f57b8070420 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.668+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b809cb60 con 0x7f57b8070420 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.672+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b804f5f0 con 0x7f57b8070420 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.672+0000 7f57affff640 1 -- 172.21.15.112:0/4202548335 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f57b00041c0 con 0x7f57b8070420 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.672+0000 7f57affff640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f57980678c0 0x7f5798069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.672+0000 7f57b6ffd640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f57980678c0 0x7f5798069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.672+0000 7f57affff640 1 -- 172.21.15.112:0/4202548335 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f57b00854f0 con 0x7f57b8070420 2024-04-25T00:55:58.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.672+0000 7f57b6ffd640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f57980678c0 0x7f5798069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57b80727a0 tx=0x7f57a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.676+0000 7f57affff640 1 -- 172.21.15.112:0/4202548335 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f57b00560a0 con 0x7f57b8070420 2024-04-25T00:55:58.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.820+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f57b80b6fd0 con 0x7f57b8070420 2024-04-25T00:55:58.821 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.820+0000 7f57affff640 1 -- 172.21.15.112:0/4202548335 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f57b00559a0 con 0x7f57b8070420 2024-04-25T00:55:58.822 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:55:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f57980678c0 msgr2=0x7f5798069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f57980678c0 0x7f5798069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57b80727a0 tx=0x7f57a8009040 comp rx=0 tx=0).stop 2024-04-25T00:55:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b8070420 msgr2=0x7f57b80708d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b8070420 0x7f57b80708d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57b80b5160 tx=0x7f57b000c890 comp rx=0 tx=0).stop 2024-04-25T00:55:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 shutdown_connections 2024-04-25T00:55:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57b8070420 0x7f57b80708d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f57980678c0 0x7f5798069d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f57b809aaa0 0x7f57b81dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 --2- 172.21.15.112:0/4202548335 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f57a4006a20 0x7f57b809d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 >> 172.21.15.112:0/4202548335 conn(0x7f57b80b8c80 msgr2=0x7f57b80b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 shutdown_connections 2024-04-25T00:55:58.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.824+0000 7f57bd73d640 1 -- 172.21.15.112:0/4202548335 wait complete. 2024-04-25T00:55:58.839 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:55:58.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f765823f640 1 Processor -- start 2024-04-25T00:55:58.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f765823f640 1 -- start start 2024-04-25T00:55:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f765823f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f765006caf0 0x7f76500b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f765823f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76500b42d0 con 0x7f76500add40 2024-04-25T00:55:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f765823f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f765009e2b0 con 0x7f765009ded0 2024-04-25T00:55:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f765823f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f76500ae790 con 0x7f765006caf0 2024-04-25T00:55:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f76557da640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f76500add40 0x7f76500b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50384/0 (socket says 172.21.15.112:50384) 2024-04-25T00:55:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f76557da640 1 -- 172.21.15.112:0/2562598921 learned_addr learned my addr 172.21.15.112:0/2562598921 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f76567dc640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f765006caf0 0x7f76500b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3061354313 0 0) 0x7f76500b42d0 con 0x7f76500add40 2024-04-25T00:55:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.960+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f763c003530 con 0x7f76500add40 2024-04-25T00:55:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1206329281 0 0) 0x7f763c003530 con 0x7f76500add40 2024-04-25T00:55:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f765006caf0 msgr2=0x7f76500b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f765006caf0 0x7f76500b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 >> v1:172.21.15.187:6789/0 conn(0x7f765009ded0 legacy=0x7f76500b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76500aea70 con 0x7f76500add40 2024-04-25T00:55:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f7640001680 con 0x7f76500add40 2024-04-25T00:55:58.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 >> v1:172.21.15.112:6789/0 conn(0x7f76500add40 legacy=0x7f76500b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c0039f0 0x7f763c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f763c004810 0x7f763c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f763c0052f0 0x7f763c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76500ae790 con 0x7f763c0039f0 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f763c00c120 con 0x7f763c004810 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f763c00c320 con 0x7f763c0052f0 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7655fdb640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c0039f0 0x7f763c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f76567dc640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f765006caf0 0x7f76500b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:58.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f76557da640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f763c004810 0x7f763c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7655fdb640 1 -- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f763c0052f0 msgr2=0x7f763c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7655fdb640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f763c0052f0 0x7f763c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7655fdb640 1 -- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f763c004810 msgr2=0x7f763c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:58.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7655fdb640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f763c004810 0x7f763c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7655fdb640 1 -- 172.21.15.112:0/2562598921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763c003530 con 0x7f763c0039f0 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7655fdb640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c0039f0 0x7f763c003e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7648001e70 tx=0x7f764801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8ecb473f4dbca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76480af070 con 0x7f763c0039f0 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76480a6040 con 0x7f763c0039f0 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f7654fd9640 1 -- 172.21.15.112:0/2562598921 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648001a90 con 0x7f763c0039f0 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f765823f640 1 -- 172.21.15.112:0/2562598921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c0039f0 msgr2=0x7f763c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f765823f640 1 --2- 172.21.15.112:0/2562598921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f763c0039f0 0x7f763c003e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7648001e70 tx=0x7f764801c780 comp rx=0 tx=0).stop 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f76567dc640 1 -- 172.21.15.112:0/2562598921 reap_dead start 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f765823f640 1 -- 172.21.15.112:0/2562598921 shutdown_connections 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f765823f640 1 -- 172.21.15.112:0/2562598921 >> 172.21.15.112:0/2562598921 conn(0x7f76500b8c80 msgr2=0x7f76500b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:58.972 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f765823f640 1 -- 172.21.15.112:0/2562598921 shutdown_connections 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.964+0000 7f765823f640 1 -- 172.21.15.112:0/2562598921 wait complete. 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 Processor -- start 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 -- start start 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f763c0039f0 0x7f7650151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76500b2460 0x7f765009b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650141540 0x7f76500b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76500ae4f0 con 0x7f7650141540 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f76500ae370 con 0x7f76500b2460 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f765014f4f0 con 0x7f763c0039f0 2024-04-25T00:55:58.973 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650141540 0x7f76500b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650141540 0x7f76500b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42054/0 (socket says 172.21.15.112:42054) 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 -- 172.21.15.112:0/3747503105 learned_addr learned my addr 172.21.15.112:0/3747503105 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76557da640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76500b2460 0x7f765009b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f7655fdb640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f763c0039f0 0x7f7650151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 -- 172.21.15.112:0/3747503105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f763c0039f0 msgr2=0x7f7650151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f763c0039f0 0x7f7650151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 -- 172.21.15.112:0/3747503105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76500b2460 msgr2=0x7f765009b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76500b2460 0x7f765009b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 -- 172.21.15.112:0/3747503105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f765009d710 con 0x7f7650141540 2024-04-25T00:55:58.974 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76567dc640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650141540 0x7f76500b0440 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f765006ced0 tx=0x7f764c00e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76467fc640 1 -- 172.21.15.112:0/3747503105 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c017030 con 0x7f7650141540 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76467fc640 1 -- 172.21.15.112:0/3747503105 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f764c004020 con 0x7f7650141540 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f765009b7e0 con 0x7f7650141540 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76467fc640 1 -- 172.21.15.112:0/3747503105 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764c00f040 con 0x7f7650141540 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f765014cae0 con 0x7f7650141540 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f765006d800 con 0x7f7650141540 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76467fc640 1 -- 172.21.15.112:0/3747503105 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f764c01a070 con 0x7f7650141540 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76467fc640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76340678f0 0x7f7634069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:58.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.968+0000 7f76467fc640 1 -- 172.21.15.112:0/3747503105 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f764c084c50 con 0x7f7650141540 2024-04-25T00:55:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.972+0000 7f7655fdb640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76340678f0 0x7f7634069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.972+0000 7f7655fdb640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76340678f0 0x7f7634069db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76480ac000 tx=0x7f7648001520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:58.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:58.976+0000 7f76467fc640 1 -- 172.21.15.112:0/3747503105 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f764c08a020 con 0x7f7650141540 2024-04-25T00:55:59.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.124+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f765004f5f0 con 0x7f7650141540 2024-04-25T00:55:59.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.124+0000 7f76467fc640 1 -- 172.21.15.112:0/3747503105 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f764c055820 con 0x7f7650141540 2024-04-25T00:55:59.125 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:55:59.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76340678f0 msgr2=0x7f7634069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76340678f0 0x7f7634069db0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76480ac000 tx=0x7f7648001520 comp rx=0 tx=0).stop 2024-04-25T00:55:59.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650141540 msgr2=0x7f76500b0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650141540 0x7f76500b0440 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f765006ced0 tx=0x7f764c00e830 comp rx=0 tx=0).stop 2024-04-25T00:55:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 shutdown_connections 2024-04-25T00:55:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7650141540 0x7f76500b0440 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76340678f0 0x7f7634069db0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76500b2460 0x7f765009b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 --2- 172.21.15.112:0/3747503105 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f763c0039f0 0x7f7650151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 >> 172.21.15.112:0/3747503105 conn(0x7f76500b8c80 msgr2=0x7f7650143a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:59.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 shutdown_connections 2024-04-25T00:55:59.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.128+0000 7f765823f640 1 -- 172.21.15.112:0/3747503105 wait complete. 2024-04-25T00:55:59.195 INFO:tasks.cephfs_test_runner:Starting test: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) Test that when cd is issued with an argument, CWD is changed 2024-04-25T00:55:59.195 INFO:tasks.cephfs_test_runner:test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-25T00:55:59.195 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:55:59.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.312+0000 7f0b48ea0640 1 Processor -- start 2024-04-25T00:55:59.314 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.312+0000 7f0b48ea0640 1 -- start start 2024-04-25T00:55:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b48ea0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b440722d0 0x7f0b44072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b48ea0640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0b44147650 0x7f0b44151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b48ea0640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b440bbec0 con 0x7f0b4409c270 2024-04-25T00:55:59.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b48ea0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0b440baaf0 con 0x7f0b440722d0 2024-04-25T00:55:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b48ea0640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0b44073100 con 0x7f0b44147650 2024-04-25T00:55:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b4259c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b440722d0 0x7f0b44072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b4259c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0b440722d0 0x7f0b44072bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42062/0 (socket says 172.21.15.112:42062) 2024-04-25T00:55:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b4259c640 1 -- 172.21.15.112:0/423145785 learned_addr learned my addr 172.21.15.112:0/423145785 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b42d9d640 1 --2- 172.21.15.112:0/423145785 >> v2:172.21.15.187:3300/0 conn(0x7f0b44147650 0x7f0b44151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b4259c640 1 -- 172.21.15.112:0/423145785 >> v2:172.21.15.187:3300/0 conn(0x7f0b44147650 msgr2=0x7f0b44151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b4259c640 1 --2- 172.21.15.112:0/423145785 >> v2:172.21.15.187:3300/0 conn(0x7f0b44147650 0x7f0b44151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b4259c640 1 -- 172.21.15.112:0/423145785 >> v1:172.21.15.187:6789/0 conn(0x7f0b4409c270 legacy=0x7f0b44147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:59.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b4259c640 1 -- 172.21.15.112:0/423145785 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b440bc5c0 con 0x7f0b440722d0 2024-04-25T00:55:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b4259c640 1 --2- 172.21.15.112:0/423145785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440722d0 0x7f0b44072bc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b4414bb80 tx=0x7f0b3401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46800bd9e71549ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.321 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b41d9b640 1 -- 172.21.15.112:0/423145785 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b340aa610 con 0x7f0b440722d0 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b41d9b640 1 -- 172.21.15.112:0/423145785 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b340af070 con 0x7f0b440722d0 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.316+0000 7f0b41d9b640 1 -- 172.21.15.112:0/423145785 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b340aa9d0 con 0x7f0b440722d0 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- 172.21.15.112:0/423145785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440722d0 msgr2=0x7f0b44072bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/423145785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440722d0 0x7f0b44072bc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b4414bb80 tx=0x7f0b3401c4b0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- 172.21.15.112:0/423145785 shutdown_connections 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/423145785 >> v2:172.21.15.187:3300/0 conn(0x7f0b44147650 0x7f0b44151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/423145785 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b440722d0 0x7f0b44072bc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- 172.21.15.112:0/423145785 >> 172.21.15.112:0/423145785 conn(0x7f0b4409e8c0 msgr2=0x7f0b4409ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- 172.21.15.112:0/423145785 shutdown_connections 2024-04-25T00:55:59.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- 172.21.15.112:0/423145785 wait complete. 2024-04-25T00:55:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 Processor -- start 2024-04-25T00:55:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- start start 2024-04-25T00:55:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b440722d0 0x7f0b440b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4409c270 0x7f0b44149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44147650 0x7f0b441428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44148410 con 0x7f0b44147650 2024-04-25T00:55:59.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4409ac50 con 0x7f0b4409c270 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0b44148590 con 0x7f0b440722d0 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44147650 0x7f0b441428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44147650 0x7f0b441428e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42064/0 (socket says 172.21.15.112:42064) 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 -- 172.21.15.112:0/4116658760 learned_addr learned my addr 172.21.15.112:0/4116658760 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b39d9b640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4409c270 0x7f0b44149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b4259c640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b440722d0 0x7f0b440b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 -- 172.21.15.112:0/4116658760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b440722d0 msgr2=0x7f0b440b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b440722d0 0x7f0b440b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 -- 172.21.15.112:0/4116658760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4409c270 msgr2=0x7f0b44149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4409c270 0x7f0b44149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 -- 172.21.15.112:0/4116658760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b440b5190 con 0x7f0b44147650 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b42d9d640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44147650 0x7f0b441428e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c007e60 tx=0x7f0b3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b3b7fe640 1 -- 172.21.15.112:0/4116658760 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c014620 con 0x7f0b44147650 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b3b7fe640 1 -- 172.21.15.112:0/4116658760 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c019070 con 0x7f0b44147650 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b440b5470 con 0x7f0b44147650 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b3b7fe640 1 -- 172.21.15.112:0/4116658760 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c014a00 con 0x7f0b44147650 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b44148770 con 0x7f0b44147650 2024-04-25T00:55:59.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.320+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b4404f580 con 0x7f0b44147650 2024-04-25T00:55:59.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.324+0000 7f0b3b7fe640 1 -- 172.21.15.112:0/4116658760 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0b3c014ba0 con 0x7f0b44147650 2024-04-25T00:55:59.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.324+0000 7f0b3b7fe640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0b240678c0 0x7f0b24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.324+0000 7f0b3b7fe640 1 -- 172.21.15.112:0/4116658760 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0b3c0865a0 con 0x7f0b44147650 2024-04-25T00:55:59.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.324+0000 7f0b4259c640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0b240678c0 0x7f0b24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.324+0000 7f0b4259c640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0b240678c0 0x7f0b24069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0b340ae000 tx=0x7f0b34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.328 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.328+0000 7f0b3b7fe640 1 -- 172.21.15.112:0/4116658760 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0b3c011260 con 0x7f0b44147650 2024-04-25T00:55:59.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.476+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b440b0300 con 0x7f0b44147650 2024-04-25T00:55:59.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b3b7fe640 1 -- 172.21.15.112:0/4116658760 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f0b3c0570e0 con 0x7f0b44147650 2024-04-25T00:55:59.478 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:55:59.478 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:55:59.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0b240678c0 msgr2=0x7f0b24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0b240678c0 0x7f0b24069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0b340ae000 tx=0x7f0b34008040 comp rx=0 tx=0).stop 2024-04-25T00:55:59.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44147650 msgr2=0x7f0b441428e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44147650 0x7f0b441428e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c007e60 tx=0x7f0b3c00e4b0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 shutdown_connections 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0b44147650 0x7f0b441428e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f0b240678c0 0x7f0b24069d80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0b4409c270 0x7f0b44149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 --2- 172.21.15.112:0/4116658760 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0b440722d0 0x7f0b440b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 >> 172.21.15.112:0/4116658760 conn(0x7f0b4409e8c0 msgr2=0x7f0b440a19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 shutdown_connections 2024-04-25T00:55:59.481 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.480+0000 7f0b48ea0640 1 -- 172.21.15.112:0/4116658760 wait complete. 2024-04-25T00:55:59.496 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:55:59.496 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:55:59.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.603+0000 7f5ebdc52640 1 Processor -- start 2024-04-25T00:55:59.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.603+0000 7f5ebdc52640 1 -- start start 2024-04-25T00:55:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5eb80b0390 0x7f5eb8148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5eb8072b20 0x7f5eb8148d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eb80b1390 con 0x7f5eb80affb0 2024-04-25T00:55:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb80b1d70 con 0x7f5eb80b0390 2024-04-25T00:55:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5eb80b1600 con 0x7f5eb8072b20 2024-04-25T00:55:59.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb6ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5eb80b0390 0x7f5eb8148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb77fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5eb80affb0 0x7f5eb809da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50386/0 (socket says 172.21.15.112:50386) 2024-04-25T00:55:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb77fe640 1 -- 172.21.15.112:0/1931526173 learned_addr learned my addr 172.21.15.112:0/1931526173 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb6ffd640 1 -- 172.21.15.112:0/1931526173 >> v2:172.21.15.187:3300/0 conn(0x7f5eb8072b20 msgr2=0x7f5eb8148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb6ffd640 1 --2- 172.21.15.112:0/1931526173 >> v2:172.21.15.187:3300/0 conn(0x7f5eb8072b20 0x7f5eb8148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb6ffd640 1 -- 172.21.15.112:0/1931526173 >> v1:172.21.15.112:6789/0 conn(0x7f5eb80affb0 legacy=0x7f5eb809da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb6ffd640 1 -- 172.21.15.112:0/1931526173 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb80b1fa0 con 0x7f5eb80b0390 2024-04-25T00:55:59.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb67fc640 1 -- 172.21.15.112:0/1931526173 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479682421 0 0) 0x7f5eb80b1390 con 0x7f5eb80affb0 2024-04-25T00:55:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb6ffd640 1 --2- 172.21.15.112:0/1931526173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb8148800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80b2bb0 tx=0x7f5ea801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a02ff720dc4fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb67fc640 1 -- 172.21.15.112:0/1931526173 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80b1070 con 0x7f5eb80b0390 2024-04-25T00:55:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb67fc640 1 -- 172.21.15.112:0/1931526173 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea80ac630 con 0x7f5eb80b0390 2024-04-25T00:55:59.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5eb67fc640 1 -- 172.21.15.112:0/1931526173 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea80a8050 con 0x7f5eb80b0390 2024-04-25T00:55:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 -- 172.21.15.112:0/1931526173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 msgr2=0x7f5eb8148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/1931526173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb8148800 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80b2bb0 tx=0x7f5ea801c3d0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 -- 172.21.15.112:0/1931526173 shutdown_connections 2024-04-25T00:55:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/1931526173 >> v2:172.21.15.187:3300/0 conn(0x7f5eb8072b20 0x7f5eb8148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/1931526173 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb8148800 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 -- 172.21.15.112:0/1931526173 >> 172.21.15.112:0/1931526173 conn(0x7f5eb80b8c80 msgr2=0x7f5eb80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:59.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 -- 172.21.15.112:0/1931526173 shutdown_connections 2024-04-25T00:55:59.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.607+0000 7f5ebdc52640 1 -- 172.21.15.112:0/1931526173 wait complete. 2024-04-25T00:55:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 Processor -- start 2024-04-25T00:55:59.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 -- start start 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5eb8072b20 0x7f5eb8141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5eb80affb0 0x7f5eb809acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb813fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb8152290 con 0x7f5eb80b0390 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5eb809e170 con 0x7f5eb80affb0 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5eb8152410 con 0x7f5eb8072b20 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb813fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb813fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42078/0 (socket says 172.21.15.112:42078) 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 -- 172.21.15.112:0/4222266271 learned_addr learned my addr 172.21.15.112:0/4222266271 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 -- 172.21.15.112:0/4222266271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5eb8072b20 msgr2=0x7f5eb8141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb77fe640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5eb8072b20 0x7f5eb8141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb6ffd640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5eb80affb0 0x7f5eb809acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5eb8072b20 0x7f5eb8141190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 -- 172.21.15.112:0/4222266271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5eb80affb0 msgr2=0x7f5eb809acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5eb80affb0 0x7f5eb809acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 -- 172.21.15.112:0/4222266271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eb80b5e90 con 0x7f5eb80b0390 2024-04-25T00:55:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5eb7fff640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb813fd60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000b9a0 tx=0x7f5eb000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5e9ffff640 1 -- 172.21.15.112:0/4222266271 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0019070 con 0x7f5eb80b0390 2024-04-25T00:55:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5e9ffff640 1 -- 172.21.15.112:0/4222266271 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb000cbc0 con 0x7f5eb80b0390 2024-04-25T00:55:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eb81409d0 con 0x7f5eb80b0390 2024-04-25T00:55:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5e9ffff640 1 -- 172.21.15.112:0/4222266271 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0014440 con 0x7f5eb80b0390 2024-04-25T00:55:59.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eb81423c0 con 0x7f5eb80b0390 2024-04-25T00:55:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eb8149550 con 0x7f5eb80b0390 2024-04-25T00:55:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.611+0000 7f5e9ffff640 1 -- 172.21.15.112:0/4222266271 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5eb001c070 con 0x7f5eb80b0390 2024-04-25T00:55:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.615+0000 7f5e9ffff640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5ea40678f0 0x7f5ea4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.615+0000 7f5e9ffff640 1 -- 172.21.15.112:0/4222266271 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5eb0085930 con 0x7f5eb80b0390 2024-04-25T00:55:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.615+0000 7f5eb77fe640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5ea40678f0 0x7f5ea4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.615+0000 7f5eb77fe640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5ea40678f0 0x7f5ea4069db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5eac000f70 tx=0x7f5eac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.619+0000 7f5e9ffff640 1 -- 172.21.15.112:0/4222266271 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5eb0056500 con 0x7f5eb80b0390 2024-04-25T00:55:59.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.767+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5eb80b1040 con 0x7f5eb80b0390 2024-04-25T00:55:59.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.767+0000 7f5e9ffff640 1 -- 172.21.15.112:0/4222266271 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f5eb0055e00 con 0x7f5eb80b0390 2024-04-25T00:55:59.768 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:55:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5ea40678f0 msgr2=0x7f5ea4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5ea40678f0 0x7f5ea4069db0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5eac000f70 tx=0x7f5eac009040 comp rx=0 tx=0).stop 2024-04-25T00:55:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 msgr2=0x7f5eb813fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb813fd60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5eb000b9a0 tx=0x7f5eb000be60 comp rx=0 tx=0).stop 2024-04-25T00:55:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 shutdown_connections 2024-04-25T00:55:59.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5eb80b0390 0x7f5eb813fd60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5ea40678f0 0x7f5ea4069db0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5eb80affb0 0x7f5eb809acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 --2- 172.21.15.112:0/4222266271 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5eb8072b20 0x7f5eb8141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 >> 172.21.15.112:0/4222266271 conn(0x7f5eb80b8c80 msgr2=0x7f5eb80b6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 shutdown_connections 2024-04-25T00:55:59.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.771+0000 7f5ebdc52640 1 -- 172.21.15.112:0/4222266271 wait complete. 2024-04-25T00:55:59.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:55:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7bab3640 1 Processor -- start 2024-04-25T00:55:59.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7bab3640 1 -- start start 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7bab3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc740adec0 0x7fcc7414ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7bab3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc740affb0 con 0x7fcc7414b240 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7bab3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc7414bb30 con 0x7fcc7414b6c0 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7bab3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc7406d630 con 0x7fcc740adec0 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7904e640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fcc7414b240 0x7fcc7406bf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50388/0 (socket says 172.21.15.112:50388) 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7904e640 1 -- 172.21.15.112:0/4047444635 learned_addr learned my addr 172.21.15.112:0/4047444635 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7984f640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc740adec0 0x7fcc7414ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343607494 0 0) 0x7fcc64001120 con 0x7fcc7414b240 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc60003530 con 0x7fcc7414b240 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3827452772 0 0) 0x7fcc60003530 con 0x7fcc7414b240 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc740adec0 msgr2=0x7fcc7414ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:59.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc740adec0 0x7fcc7414ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 >> v1:172.21.15.187:6789/0 conn(0x7fcc7414b6c0 legacy=0x7fcc74151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc740af2f0 con 0x7fcc7414b240 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7fcc64001850 con 0x7fcc7414b240 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 >> v1:172.21.15.112:6789/0 conn(0x7fcc7414b240 legacy=0x7fcc7406bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc600039f0 0x7fcc60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc60004810 0x7fcc60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc740adec0 0x7fcc600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc7406d630 con 0x7fcc60004810 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6000ba80 con 0x7fcc600039f0 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6000bc80 con 0x7fcc740adec0 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7904e640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc60004810 0x7fcc60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7984f640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc600039f0 0x7fcc60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7904e640 1 -- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc740adec0 msgr2=0x7fcc600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:59.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7904e640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc740adec0 0x7fcc600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7904e640 1 -- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc600039f0 msgr2=0x7fcc60003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:55:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7904e640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc600039f0 0x7fcc60003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.903+0000 7fcc7904e640 1 -- 172.21.15.112:0/4047444635 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc60003530 con 0x7fcc60004810 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc60004810 0x7fcc60004c90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400c490 tx=0x7fcc6401f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2644b2b0374ff89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6400c0e0 con 0x7fcc60004810 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc64005b00 con 0x7fcc60004810 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7884d640 1 -- 172.21.15.112:0/4047444635 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64007070 con 0x7fcc60004810 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- 172.21.15.112:0/4047444635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc60004810 msgr2=0x7fcc60004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 --2- 172.21.15.112:0/4047444635 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc60004810 0x7fcc60004c90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400c490 tx=0x7fcc6401f3a0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7a050640 1 -- 172.21.15.112:0/4047444635 reap_dead start 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- 172.21.15.112:0/4047444635 shutdown_connections 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- 172.21.15.112:0/4047444635 >> 172.21.15.112:0/4047444635 conn(0x7fcc740b8c80 msgr2=0x7fcc740b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:55:59.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- 172.21.15.112:0/4047444635 shutdown_connections 2024-04-25T00:55:59.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- 172.21.15.112:0/4047444635 wait complete. 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 Processor -- start 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- start start 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc60004810 0x7fcc741d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc741e03b0 0x7fcc741e0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc74143300 0x7fcc74143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740b4180 con 0x7fcc741e03b0 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fcc740b2f30 con 0x7fcc60004810 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7bab3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fcc740ae2a0 con 0x7fcc74143300 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc741e03b0 0x7fcc741e0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc741e03b0 0x7fcc741e0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42090/0 (socket says 172.21.15.112:42090) 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 -- 172.21.15.112:0/3115764878 learned_addr learned my addr 172.21.15.112:0/3115764878 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7984f640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc60004810 0x7fcc741d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7a050640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc74143300 0x7fcc74143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 -- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc74143300 msgr2=0x7fcc74143780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc74143300 0x7fcc74143780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 -- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc60004810 msgr2=0x7fcc741d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc60004810 0x7fcc741d2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 -- 172.21.15.112:0/3115764878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc741466c0 con 0x7fcc741e03b0 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7a050640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc74143300 0x7fcc74143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7984f640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc60004810 0x7fcc741d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc7904e640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc741e03b0 0x7fcc741e0830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400c490 tx=0x7fcc60005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc6a7fc640 1 -- 172.21.15.112:0/3115764878 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64001120 con 0x7fcc741e03b0 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.907+0000 7fcc6a7fc640 1 -- 172.21.15.112:0/3115764878 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc640047f0 con 0x7fcc741e03b0 2024-04-25T00:55:59.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc74145390 con 0x7fcc741e03b0 2024-04-25T00:55:59.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc6a7fc640 1 -- 172.21.15.112:0/3115764878 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc64004ba0 con 0x7fcc741e03b0 2024-04-25T00:55:59.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc741d3620 con 0x7fcc741e03b0 2024-04-25T00:55:59.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc74145560 con 0x7fcc741e03b0 2024-04-25T00:55:59.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc6a7fc640 1 -- 172.21.15.112:0/3115764878 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcc64007070 con 0x7fcc741e03b0 2024-04-25T00:55:59.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc6a7fc640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc580678c0 0x7fcc58069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:55:59.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc6a7fc640 1 -- 172.21.15.112:0/3115764878 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fcc641230f0 con 0x7fcc741e03b0 2024-04-25T00:55:59.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc7984f640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc580678c0 0x7fcc58069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:55:59.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.911+0000 7fcc7984f640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc580678c0 0x7fcc58069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc740b0280 tx=0x7fcc70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:55:59.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:55:59.915+0000 7fcc6a7fc640 1 -- 172.21.15.112:0/3115764878 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcc740affb0 con 0x7fcc741e03b0 2024-04-25T00:56:00.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.075+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcc7404f5f0 con 0x7fcc741e03b0 2024-04-25T00:56:00.076 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.075+0000 7fcc6a7fc640 1 -- 172.21.15.112:0/3115764878 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fcc640f3c30 con 0x7fcc741e03b0 2024-04-25T00:56:00.076 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:56:00.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc580678c0 msgr2=0x7fcc58069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc580678c0 0x7fcc58069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc740b0280 tx=0x7fcc70008040 comp rx=0 tx=0).stop 2024-04-25T00:56:00.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc741e03b0 msgr2=0x7fcc741e0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.080 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc741e03b0 0x7fcc741e0830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc6400c490 tx=0x7fcc60005840 comp rx=0 tx=0).stop 2024-04-25T00:56:00.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 shutdown_connections 2024-04-25T00:56:00.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fcc74143300 0x7fcc74143780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc741e03b0 0x7fcc741e0830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fcc580678c0 0x7fcc58069d80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 --2- 172.21.15.112:0/3115764878 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fcc60004810 0x7fcc741d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 >> 172.21.15.112:0/3115764878 conn(0x7fcc740b8c80 msgr2=0x7fcc7406cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:00.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 shutdown_connections 2024-04-25T00:56:00.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.079+0000 7fcc7bab3640 1 -- 172.21.15.112:0/3115764878 wait complete. 2024-04-25T00:56:00.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:56:00.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdafe20e640 1 Processor -- start 2024-04-25T00:56:00.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdafe20e640 1 -- start start 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdafe20e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdaf80742f0 0x7fdaf806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdafe20e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf8073530 0x7fdaf80716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdafe20e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf8148a60 con 0x7fdaf80746d0 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdafe20e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf8070270 con 0x7fdaf80742f0 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdafe20e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf8148810 con 0x7fdaf8073530 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdaf80742f0 0x7fdaf806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdaf80742f0 0x7fdaf806fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42106/0 (socket says 172.21.15.112:42106) 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf6ffd640 1 --1- 172.21.15.112:0/1597091659 >> v1:172.21.15.112:6789/0 conn(0x7fdaf80746d0 0x7fdaf813f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50402/0 (socket says 172.21.15.112:50402) 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf77fe640 1 -- 172.21.15.112:0/1597091659 learned_addr learned my addr 172.21.15.112:0/1597091659 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:00.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf7fff640 1 --2- 172.21.15.112:0/1597091659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf8073530 0x7fdaf80716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf77fe640 1 -- 172.21.15.112:0/1597091659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf8073530 msgr2=0x7fdaf80716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf77fe640 1 --2- 172.21.15.112:0/1597091659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf8073530 0x7fdaf80716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf77fe640 1 -- 172.21.15.112:0/1597091659 >> v1:172.21.15.112:6789/0 conn(0x7fdaf80746d0 legacy=0x7fdaf813f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf67fc640 1 -- 172.21.15.112:0/1597091659 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 471138019 0 0) 0x7fdaf8148a60 con 0x7fdaf80746d0 2024-04-25T00:56:00.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.219+0000 7fdaf77fe640 1 -- 172.21.15.112:0/1597091659 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf8148d40 con 0x7fdaf80742f0 2024-04-25T00:56:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 --2- 172.21.15.112:0/1597091659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf80742f0 0x7fdaf806fc90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009870 tx=0x7fdae801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4b87354808da10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf67fc640 1 -- 172.21.15.112:0/1597091659 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80aa620 con 0x7fdaf80742f0 2024-04-25T00:56:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf67fc640 1 -- 172.21.15.112:0/1597091659 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae80af070 con 0x7fdaf80742f0 2024-04-25T00:56:00.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf67fc640 1 -- 172.21.15.112:0/1597091659 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80aa9e0 con 0x7fdaf80742f0 2024-04-25T00:56:00.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- 172.21.15.112:0/1597091659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf80742f0 msgr2=0x7fdaf806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 --2- 172.21.15.112:0/1597091659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf80742f0 0x7fdaf806fc90 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009870 tx=0x7fdae801c560 comp rx=0 tx=0).stop 2024-04-25T00:56:00.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- 172.21.15.112:0/1597091659 shutdown_connections 2024-04-25T00:56:00.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 --2- 172.21.15.112:0/1597091659 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf8073530 0x7fdaf80716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 --2- 172.21.15.112:0/1597091659 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf80742f0 0x7fdaf806fc90 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- 172.21.15.112:0/1597091659 >> 172.21.15.112:0/1597091659 conn(0x7fdaf80b91d0 msgr2=0x7fdaf80b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:00.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- 172.21.15.112:0/1597091659 shutdown_connections 2024-04-25T00:56:00.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- 172.21.15.112:0/1597091659 wait complete. 2024-04-25T00:56:00.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 Processor -- start 2024-04-25T00:56:00.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- start start 2024-04-25T00:56:00.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf8073530 0x7fdaf81e11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf80742f0 0x7fdaf8141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf80746d0 0x7fdaf8140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf81525d0 con 0x7fdaf8073530 2024-04-25T00:56:00.226 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf80b36c0 con 0x7fdaf80742f0 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf8152750 con 0x7fdaf80746d0 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf8073530 0x7fdaf81e11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf8073530 0x7fdaf81e11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42108/0 (socket says 172.21.15.112:42108) 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 -- 172.21.15.112:0/86035087 learned_addr learned my addr 172.21.15.112:0/86035087 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf6ffd640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf80742f0 0x7fdaf8141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf7fff640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf80746d0 0x7fdaf8140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 -- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf80746d0 msgr2=0x7fdaf8140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf80746d0 0x7fdaf8140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 -- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf80742f0 msgr2=0x7fdaf8141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf80742f0 0x7fdaf8141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 -- 172.21.15.112:0/86035087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf8142970 con 0x7fdaf8073530 2024-04-25T00:56:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf7fff640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf80746d0 0x7fdaf8140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaf77fe640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf8073530 0x7fdaf81e11b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdae80ac040 tx=0x7fdae801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaeffff640 1 -- 172.21.15.112:0/86035087 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80aad90 con 0x7fdaf8073530 2024-04-25T00:56:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaeffff640 1 -- 172.21.15.112:0/86035087 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae80bd3f0 con 0x7fdaf8073530 2024-04-25T00:56:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf813dd10 con 0x7fdaf8073530 2024-04-25T00:56:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdaeffff640 1 -- 172.21.15.112:0/86035087 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae80bd7e0 con 0x7fdaf8073530 2024-04-25T00:56:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.223+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf813e190 con 0x7fdaf8073530 2024-04-25T00:56:00.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.227+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf81e1ee0 con 0x7fdaf8073530 2024-04-25T00:56:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.227+0000 7fdaeffff640 1 -- 172.21.15.112:0/86035087 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdae80c7420 con 0x7fdaf8073530 2024-04-25T00:56:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.227+0000 7fdaeffff640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdad80678c0 0x7fdad8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.227+0000 7fdaeffff640 1 -- 172.21.15.112:0/86035087 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fdae8120170 con 0x7fdaf8073530 2024-04-25T00:56:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.227+0000 7fdaf6ffd640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdad80678c0 0x7fdad8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.227+0000 7fdaf6ffd640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdad80678c0 0x7fdad8069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80b46a0 tx=0x7fdae0007ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:00.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.231+0000 7fdaeffff640 1 -- 172.21.15.112:0/86035087 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdae80c3060 con 0x7fdaf8073530 2024-04-25T00:56:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.379+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdaf80b8ad0 con 0x7fdaf8073530 2024-04-25T00:56:00.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.379+0000 7fdaeffff640 1 -- 172.21.15.112:0/86035087 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fdae8016150 con 0x7fdaf8073530 2024-04-25T00:56:00.381 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdad80678c0 msgr2=0x7fdad8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdad80678c0 0x7fdad8069d80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdaf80b46a0 tx=0x7fdae0007ba0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf8073530 msgr2=0x7fdaf81e11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf8073530 0x7fdaf81e11b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdae80ac040 tx=0x7fdae801ada0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 shutdown_connections 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdaf80746d0 0x7fdaf8140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdad80678c0 0x7fdad8069d80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdaf80742f0 0x7fdaf8141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 --2- 172.21.15.112:0/86035087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdaf8073530 0x7fdaf81e11b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 >> 172.21.15.112:0/86035087 conn(0x7fdaf80b91d0 msgr2=0x7fdaf813b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 shutdown_connections 2024-04-25T00:56:00.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.383+0000 7fdafe20e640 1 -- 172.21.15.112:0/86035087 wait complete. 2024-04-25T00:56:00.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-04-25T00:56:00.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.519+0000 7f5bf77c2640 1 Processor -- start 2024-04-25T00:56:00.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.519+0000 7f5bf77c2640 1 -- start start 2024-04-25T00:56:00.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf77c2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf0074590 0x7f5bf01442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf77c2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bf0144830 0x7f5bf00b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf77c2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf00ae100 con 0x7f5bf00b5200 2024-04-25T00:56:00.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf77c2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf0074970 con 0x7f5bf0144830 2024-04-25T00:56:00.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf77c2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf00af740 con 0x7f5bf0074590 2024-04-25T00:56:00.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf4d5d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bf0144830 0x7f5bf00b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf4d5d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5bf0144830 0x7f5bf00b4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42122/0 (socket says 172.21.15.112:42122) 2024-04-25T00:56:00.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf4d5d640 1 -- 172.21.15.112:0/44457955 learned_addr learned my addr 172.21.15.112:0/44457955 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:00.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf555e640 1 --2- 172.21.15.112:0/44457955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf0074590 0x7f5bf01442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf4d5d640 1 -- 172.21.15.112:0/44457955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf0074590 msgr2=0x7f5bf01442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:00.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf4d5d640 1 --2- 172.21.15.112:0/44457955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf0074590 0x7f5bf01442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf4d5d640 1 -- 172.21.15.112:0/44457955 >> v1:172.21.15.187:6789/0 conn(0x7f5bf00b5200 legacy=0x7f5bf0147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:00.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf4d5d640 1 -- 172.21.15.112:0/44457955 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf00afeb0 con 0x7f5bf0144830 2024-04-25T00:56:00.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5bf4d5d640 1 --2- 172.21.15.112:0/44457955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0144830 0x7f5bf00b4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009870 tx=0x7f5be001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841256030f52b8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:00.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.523+0000 7f5be7fff640 1 -- 172.21.15.112:0/44457955 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00ab640 con 0x7f5bf0144830 2024-04-25T00:56:00.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5be7fff640 1 -- 172.21.15.112:0/44457955 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be00b0070 con 0x7f5bf0144830 2024-04-25T00:56:00.526 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5be7fff640 1 -- 172.21.15.112:0/44457955 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00aba00 con 0x7f5bf0144830 2024-04-25T00:56:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- 172.21.15.112:0/44457955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0144830 msgr2=0x7f5bf00b4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/44457955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0144830 0x7f5bf00b4cc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009870 tx=0x7f5be001c460 comp rx=0 tx=0).stop 2024-04-25T00:56:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- 172.21.15.112:0/44457955 shutdown_connections 2024-04-25T00:56:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/44457955 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0144830 0x7f5bf00b4cc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/44457955 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf0074590 0x7f5bf01442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- 172.21.15.112:0/44457955 >> 172.21.15.112:0/44457955 conn(0x7f5bf00b9700 msgr2=0x7f5bf00b9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- 172.21.15.112:0/44457955 shutdown_connections 2024-04-25T00:56:00.527 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- 172.21.15.112:0/44457955 wait complete. 2024-04-25T00:56:00.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 Processor -- start 2024-04-25T00:56:00.528 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- start start 2024-04-25T00:56:00.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0074590 0x7f5bf0143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf00b5200 0x7f5bf01e0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bf0144830 0x7f5bf013d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf00af030 con 0x7f5bf0074590 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf00ae890 con 0x7f5bf0144830 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf0071000 con 0x7f5bf00b5200 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0074590 0x7f5bf0143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0074590 0x7f5bf0143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42126/0 (socket says 172.21.15.112:42126) 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 -- 172.21.15.112:0/3508732051 learned_addr learned my addr 172.21.15.112:0/3508732051 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf5d5f640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bf0144830 0x7f5bf013d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf4d5d640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf00b5200 0x7f5bf01e0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.530 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 -- 172.21.15.112:0/3508732051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf00b5200 msgr2=0x7f5bf01e0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf00b5200 0x7f5bf01e0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 -- 172.21.15.112:0/3508732051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bf0144830 msgr2=0x7f5bf013d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bf0144830 0x7f5bf013d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 -- 172.21.15.112:0/3508732051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf00712e0 con 0x7f5bf0074590 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf555e640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0074590 0x7f5bf0143760 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5be800cdf0 tx=0x7f5be800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5be5ffb640 1 -- 172.21.15.112:0/3508732051 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8003ba0 con 0x7f5bf0074590 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5be5ffb640 1 -- 172.21.15.112:0/3508732051 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be8003d60 con 0x7f5bf0074590 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5be5ffb640 1 -- 172.21.15.112:0/3508732051 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be8004260 con 0x7f5bf0074590 2024-04-25T00:56:00.531 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf00b1870 con 0x7f5bf0074590 2024-04-25T00:56:00.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.527+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf0144390 con 0x7f5bf0074590 2024-04-25T00:56:00.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.531+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf0070e20 con 0x7f5bf0074590 2024-04-25T00:56:00.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.531+0000 7f5be5ffb640 1 -- 172.21.15.112:0/3508732051 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5be8004400 con 0x7f5bf0074590 2024-04-25T00:56:00.532 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.531+0000 7f5be5ffb640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5bd40678f0 0x7f5bd4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:00.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.531+0000 7f5be5ffb640 1 -- 172.21.15.112:0/3508732051 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5be80867d0 con 0x7f5bf0074590 2024-04-25T00:56:00.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.531+0000 7f5bf4d5d640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5bd40678f0 0x7f5bd4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:00.533 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.531+0000 7f5bf4d5d640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5bd40678f0 0x7f5bd4069db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5be00af000 tx=0x7f5be00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:00.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.535+0000 7f5be5ffb640 1 -- 172.21.15.112:0/3508732051 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5bf0070e20 con 0x7f5bf0074590 2024-04-25T00:56:00.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:00.755+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7f5bf004f580 con 0x7f5bf0074590 2024-04-25T00:56:01.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.743+0000 7f5be5ffb640 1 -- 172.21.15.112:0/3508732051 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v45) ==== 137+0+0 (secure 0 0 0) 0x7f5be80573a0 con 0x7f5bf0074590 2024-04-25T00:56:01.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5bd40678f0 msgr2=0x7f5bd4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:01.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5bd40678f0 0x7f5bd4069db0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5be00af000 tx=0x7f5be00a7000 comp rx=0 tx=0).stop 2024-04-25T00:56:01.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0074590 msgr2=0x7f5bf0143760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:01.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0074590 0x7f5bf0143760 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5be800cdf0 tx=0x7f5be800a660 comp rx=0 tx=0).stop 2024-04-25T00:56:01.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 shutdown_connections 2024-04-25T00:56:01.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5bd40678f0 0x7f5bd4069db0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5bf0144830 0x7f5bf013d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5bf00b5200 0x7f5bf01e0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 --2- 172.21.15.112:0/3508732051 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5bf0074590 0x7f5bf0143760 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 >> 172.21.15.112:0/3508732051 conn(0x7f5bf00b9700 msgr2=0x7f5bf00b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 shutdown_connections 2024-04-25T00:56:01.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.747+0000 7f5bf77c2640 1 -- 172.21.15.112:0/3508732051 wait complete. 2024-04-25T00:56:01.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T00:56:01.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.895+0000 7f1065435640 1 Processor -- start 2024-04-25T00:56:01.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.895+0000 7f1065435640 1 -- start start 2024-04-25T00:56:01.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f1065435640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10600b9030 0x7f106009acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:01.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f1065435640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106009b800 0x7f1060141350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:01.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f1065435640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10600b9410 con 0x7f1060141890 2024-04-25T00:56:01.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f1065435640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f10600b9990 con 0x7f10600b9030 2024-04-25T00:56:01.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f1065435640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f10600baa90 con 0x7f106009b800 2024-04-25T00:56:01.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105effd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f10600b9030 0x7f106009acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:01.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105f7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1060141890 0x7f1060072360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50408/0 (socket says 172.21.15.112:50408) 2024-04-25T00:56:01.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105f7fe640 1 -- 172.21.15.112:0/94026651 learned_addr learned my addr 172.21.15.112:0/94026651 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:01.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105e7fc640 1 --2- 172.21.15.112:0/94026651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106009b800 0x7f1060141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:01.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105effd640 1 -- 172.21.15.112:0/94026651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106009b800 msgr2=0x7f1060141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:01.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105effd640 1 --2- 172.21.15.112:0/94026651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106009b800 0x7f1060141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105effd640 1 -- 172.21.15.112:0/94026651 >> v1:172.21.15.112:6789/0 conn(0x7f1060141890 legacy=0x7f1060072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:01.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105effd640 1 -- 172.21.15.112:0/94026651 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10600bad70 con 0x7f10600b9030 2024-04-25T00:56:01.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105dffb640 1 -- 172.21.15.112:0/94026651 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1556136489 0 0) 0x7f10600b9410 con 0x7f1060141890 2024-04-25T00:56:01.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105e7fc640 1 --2- 172.21.15.112:0/94026651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106009b800 0x7f1060141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105effd640 1 --2- 172.21.15.112:0/94026651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f106009acb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1058009d90 tx=0x7f105801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0453a0140b4ff78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105dffb640 1 -- 172.21.15.112:0/94026651 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10580b2070 con 0x7f10600b9030 2024-04-25T00:56:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105dffb640 1 -- 172.21.15.112:0/94026651 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10580a5770 con 0x7f10600b9030 2024-04-25T00:56:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.899+0000 7f105dffb640 1 -- 172.21.15.112:0/94026651 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10580a5b40 con 0x7f10600b9030 2024-04-25T00:56:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- 172.21.15.112:0/94026651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 msgr2=0x7f106009acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:01.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 --2- 172.21.15.112:0/94026651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f106009acb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1058009d90 tx=0x7f105801c8e0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- 172.21.15.112:0/94026651 shutdown_connections 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 --2- 172.21.15.112:0/94026651 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f106009b800 0x7f1060141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 --2- 172.21.15.112:0/94026651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f106009acb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- 172.21.15.112:0/94026651 >> 172.21.15.112:0/94026651 conn(0x7f106009eaf0 msgr2=0x7f106009ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- 172.21.15.112:0/94026651 shutdown_connections 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- 172.21.15.112:0/94026651 wait complete. 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 Processor -- start 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- start start 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106009b800 0x7f106009d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f10601e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1060141890 0x7f10601e3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f106009b370 con 0x7f10600b9030 2024-04-25T00:56:01.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f106009b1f0 con 0x7f106009b800 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f106009c140 con 0x7f1060141890 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f10601e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f10601e2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42142/0 (socket says 172.21.15.112:42142) 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 -- 172.21.15.112:0/2985309383 learned_addr learned my addr 172.21.15.112:0/2985309383 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105f7fe640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1060141890 0x7f10601e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 -- 172.21.15.112:0/2985309383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1060141890 msgr2=0x7f10601e3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105effd640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106009b800 0x7f106009d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1060141890 0x7f10601e3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 -- 172.21.15.112:0/2985309383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106009b800 msgr2=0x7f106009d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106009b800 0x7f106009d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 -- 172.21.15.112:0/2985309383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10600b6250 con 0x7f10600b9030 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f105e7fc640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f10601e2fd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f104800c7f0 tx=0x7f104800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f10577fe640 1 -- 172.21.15.112:0/2985309383 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048015070 con 0x7f10600b9030 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f10577fe640 1 -- 172.21.15.112:0/2985309383 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1048007a30 con 0x7f10600b9030 2024-04-25T00:56:01.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10600b6530 con 0x7f10600b9030 2024-04-25T00:56:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f10577fe640 1 -- 172.21.15.112:0/2985309383 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048007e00 con 0x7f10600b9030 2024-04-25T00:56:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10600b6dd0 con 0x7f10600b9030 2024-04-25T00:56:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.903+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f106004f5f0 con 0x7f10600b9030 2024-04-25T00:56:01.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.907+0000 7f10577fe640 1 -- 172.21.15.112:0/2985309383 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1048018070 con 0x7f10600b9030 2024-04-25T00:56:01.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.907+0000 7f10577fe640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10400678c0 0x7f1040069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:01.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.907+0000 7f105effd640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10400678c0 0x7f1040069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:01.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.907+0000 7f10577fe640 1 -- 172.21.15.112:0/2985309383 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1048085bf0 con 0x7f10600b9030 2024-04-25T00:56:01.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.907+0000 7f105effd640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10400678c0 0x7f1040069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10580af000 tx=0x7f10580a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:01.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:01.911+0000 7f10577fe640 1 -- 172.21.15.112:0/2985309383 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1048056740 con 0x7f10600b9030 2024-04-25T00:56:02.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.059+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f10600b1da0 con 0x7f10600b9030 2024-04-25T00:56:02.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.063+0000 7f10577fe640 1 -- 172.21.15.112:0/2985309383 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f1048056040 con 0x7f10600b9030 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10400678c0 msgr2=0x7f1040069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10400678c0 0x7f1040069d80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10580af000 tx=0x7f10580a7070 comp rx=0 tx=0).stop 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 msgr2=0x7f10601e2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f10601e2fd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f104800c7f0 tx=0x7f104800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 shutdown_connections 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1060141890 0x7f10601e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f10600b9030 0x7f10601e2fd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f10400678c0 0x7f1040069d80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 --2- 172.21.15.112:0/2985309383 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f106009b800 0x7f106009d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 >> 172.21.15.112:0/2985309383 conn(0x7f106009eaf0 msgr2=0x7f10600bd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:02.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 shutdown_connections 2024-04-25T00:56:02.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.067+0000 7f1065435640 1 -- 172.21.15.112:0/2985309383 wait complete. 2024-04-25T00:56:02.084 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:56:02.084 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:56:02.135 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:56:02.195 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:56:02.195 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:56:02.195 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:56:02.201 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:56:02.201 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:56:02.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.395+0000 7f7e9811e640 1 Processor -- start 2024-04-25T00:56:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.395+0000 7f7e9811e640 1 -- start start 2024-04-25T00:56:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.395+0000 7f7e9811e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7e900722d0 0x7f7e90072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.395+0000 7f7e9811e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e90147650 0x7f7e90151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.395+0000 7f7e9811e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e900bbec0 con 0x7f7e9009c270 2024-04-25T00:56:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.395+0000 7f7e9811e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7e900baaf0 con 0x7f7e90147650 2024-04-25T00:56:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.395+0000 7f7e9811e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7e90073100 con 0x7f7e900722d0 2024-04-25T00:56:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.395+0000 7f7e966bb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7e90147650 0x7f7e90151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e956b9640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7e9009c270 0x7f7e90147d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50420/0 (socket says 172.21.15.112:50420) 2024-04-25T00:56:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e956b9640 1 -- 172.21.15.112:0/2756284134 learned_addr learned my addr 172.21.15.112:0/2756284134 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e966bb640 1 -- 172.21.15.112:0/2756284134 >> v2:172.21.15.187:3300/0 conn(0x7f7e900722d0 msgr2=0x7f7e90072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e95eba640 1 --2- 172.21.15.112:0/2756284134 >> v2:172.21.15.187:3300/0 conn(0x7f7e900722d0 0x7f7e90072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e966bb640 1 --2- 172.21.15.112:0/2756284134 >> v2:172.21.15.187:3300/0 conn(0x7f7e900722d0 0x7f7e90072bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e966bb640 1 -- 172.21.15.112:0/2756284134 >> v1:172.21.15.112:6789/0 conn(0x7f7e9009c270 legacy=0x7f7e90147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e966bb640 1 -- 172.21.15.112:0/2756284134 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e900bc5c0 con 0x7f7e90147650 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e966bb640 1 --2- 172.21.15.112:0/2756284134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90147650 0x7f7e90151ef0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e900ba1f0 tx=0x7f7e8801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae14d33afa56e73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e94eb8640 1 -- 172.21.15.112:0/2756284134 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880b2070 con 0x7f7e90147650 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e94eb8640 1 -- 172.21.15.112:0/2756284134 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e880a59c0 con 0x7f7e90147650 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e94eb8640 1 -- 172.21.15.112:0/2756284134 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880a5d90 con 0x7f7e90147650 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e9811e640 1 -- 172.21.15.112:0/2756284134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90147650 msgr2=0x7f7e90151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e9811e640 1 --2- 172.21.15.112:0/2756284134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90147650 0x7f7e90151ef0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e900ba1f0 tx=0x7f7e8801c900 comp rx=0 tx=0).stop 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e9811e640 1 -- 172.21.15.112:0/2756284134 shutdown_connections 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e9811e640 1 --2- 172.21.15.112:0/2756284134 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e90147650 0x7f7e90151ef0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e9811e640 1 --2- 172.21.15.112:0/2756284134 >> v2:172.21.15.187:3300/0 conn(0x7f7e900722d0 0x7f7e90072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e9811e640 1 -- 172.21.15.112:0/2756284134 >> 172.21.15.112:0/2756284134 conn(0x7f7e9009e8c0 msgr2=0x7f7e9009ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e9811e640 1 -- 172.21.15.112:0/2756284134 shutdown_connections 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.399+0000 7f7e9811e640 1 -- 172.21.15.112:0/2756284134 wait complete. 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 Processor -- start 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 -- start start 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e900722d0 0x7f7e901e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9009c270 0x7f7e900b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e90147650 0x7f7e900b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7e90148410 con 0x7f7e9009c270 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9009ac50 con 0x7f7e900722d0 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7e90148590 con 0x7f7e90147650 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9009c270 0x7f7e900b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9009c270 0x7f7e900b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42168/0 (socket says 172.21.15.112:42168) 2024-04-25T00:56:02.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 -- 172.21.15.112:0/1761945041 learned_addr learned my addr 172.21.15.112:0/1761945041 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e95eba640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e900722d0 0x7f7e901e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 -- 172.21.15.112:0/1761945041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e90147650 msgr2=0x7f7e900b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e90147650 0x7f7e900b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 -- 172.21.15.112:0/1761945041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e900722d0 msgr2=0x7f7e901e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e900722d0 0x7f7e901e3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 -- 172.21.15.112:0/1761945041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e900a1d50 con 0x7f7e9009c270 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e956b9640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9009c270 0x7f7e900b0db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c420 tx=0x7f7e8000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e867fc640 1 -- 172.21.15.112:0/1761945041 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80017070 con 0x7f7e9009c270 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e867fc640 1 -- 172.21.15.112:0/1761945041 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80005cf0 con 0x7f7e9009c270 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e901e3bd0 con 0x7f7e9009c270 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e867fc640 1 -- 172.21.15.112:0/1761945041 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e800072f0 con 0x7f7e9009c270 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e900a1fc0 con 0x7f7e9009c270 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e9004f580 con 0x7f7e9009c270 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e867fc640 1 -- 172.21.15.112:0/1761945041 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7e8001a070 con 0x7f7e9009c270 2024-04-25T00:56:02.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e867fc640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7e740678c0 0x7f7e74069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e95eba640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7e740678c0 0x7f7e74069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e867fc640 1 -- 172.21.15.112:0/1761945041 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7e80084fb0 con 0x7f7e9009c270 2024-04-25T00:56:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.403+0000 7f7e95eba640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7e740678c0 0x7f7e74069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00b7c0 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.407+0000 7f7e867fc640 1 -- 172.21.15.112:0/1761945041 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7e80055b80 con 0x7f7e9009c270 2024-04-25T00:56:02.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.559+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e9009f060 con 0x7f7e9009c270 2024-04-25T00:56:02.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.559+0000 7f7e867fc640 1 -- 172.21.15.112:0/1761945041 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f7e80055480 con 0x7f7e9009c270 2024-04-25T00:56:02.561 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:02.561 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7e740678c0 msgr2=0x7f7e74069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7e740678c0 0x7f7e74069d80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00b7c0 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2024-04-25T00:56:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9009c270 msgr2=0x7f7e900b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9009c270 0x7f7e900b0db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7e8000c420 tx=0x7f7e8000c8e0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 shutdown_connections 2024-04-25T00:56:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7e90147650 0x7f7e900b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e9009c270 0x7f7e900b0db0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7e740678c0 0x7f7e74069d80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 --2- 172.21.15.112:0/1761945041 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7e900722d0 0x7f7e901e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 >> 172.21.15.112:0/1761945041 conn(0x7f7e9009e8c0 msgr2=0x7f7e900b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 shutdown_connections 2024-04-25T00:56:02.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.563+0000 7f7e9811e640 1 -- 172.21.15.112:0/1761945041 wait complete. 2024-04-25T00:56:02.579 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:56:02.579 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:56:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4ea33640 1 Processor -- start 2024-04-25T00:56:02.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4ea33640 1 -- start start 2024-04-25T00:56:02.691 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4ea33640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6f48150f00 0x7f6f481532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4ea33640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f480bba90 con 0x7f6f480723f0 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4ea33640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f480b9be0 con 0x7f6f480a05a0 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4ea33640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6f480ba210 con 0x7f6f48150f00 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4cfd0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6f48150f00 0x7f6f481532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f47fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6f480723f0 0x7f6f480b94d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:50430/0 (socket says 172.21.15.112:50430) 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f47fff640 1 -- 172.21.15.112:0/1064195401 learned_addr learned my addr 172.21.15.112:0/1064195401 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4cfd0640 1 -- 172.21.15.112:0/1064195401 >> v1:172.21.15.187:6789/0 conn(0x7f6f480a05a0 legacy=0x7f6f48141590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4cfd0640 1 -- 172.21.15.112:0/1064195401 >> v1:172.21.15.112:6789/0 conn(0x7f6f480723f0 legacy=0x7f6f480b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.691+0000 7f6f4cfd0640 1 -- 172.21.15.112:0/1064195401 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f480727d0 con 0x7f6f48150f00 2024-04-25T00:56:02.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 --2- 172.21.15.112:0/1064195401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481532f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6f480ba390 tx=0x7f6f4001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a8e929942459c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.694 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f46ffd640 1 -- 172.21.15.112:0/1064195401 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400b2070 con 0x7f6f48150f00 2024-04-25T00:56:02.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f46ffd640 1 -- 172.21.15.112:0/1064195401 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f400a5830 con 0x7f6f48150f00 2024-04-25T00:56:02.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f46ffd640 1 -- 172.21.15.112:0/1064195401 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400a5c00 con 0x7f6f48150f00 2024-04-25T00:56:02.695 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- 172.21.15.112:0/1064195401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 msgr2=0x7f6f481532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 --2- 172.21.15.112:0/1064195401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481532f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6f480ba390 tx=0x7f6f4001c880 comp rx=0 tx=0).stop 2024-04-25T00:56:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- 172.21.15.112:0/1064195401 shutdown_connections 2024-04-25T00:56:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 --2- 172.21.15.112:0/1064195401 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481532f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- 172.21.15.112:0/1064195401 >> 172.21.15.112:0/1064195401 conn(0x7f6f480b75d0 msgr2=0x7f6f480b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- 172.21.15.112:0/1064195401 shutdown_connections 2024-04-25T00:56:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- 172.21.15.112:0/1064195401 wait complete. 2024-04-25T00:56:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 Processor -- start 2024-04-25T00:56:02.696 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- start start 2024-04-25T00:56:02.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f480723f0 0x7f6f48139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f480a05a0 0x7f6f48140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6f480b8ab0 con 0x7f6f48150f00 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6f480b8930 con 0x7f6f480723f0 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6f48141ca0 con 0x7f6f480a05a0 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:42186/0 (socket says 172.21.15.112:42186) 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 -- 172.21.15.112:0/3540254416 learned_addr learned my addr 172.21.15.112:0/3540254416 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f47fff640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f480723f0 0x7f6f48139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f477fe640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f480a05a0 0x7f6f48140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 -- 172.21.15.112:0/3540254416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f480a05a0 msgr2=0x7f6f48140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f480a05a0 0x7f6f48140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.699 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 -- 172.21.15.112:0/3540254416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f480723f0 msgr2=0x7f6f48139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f480723f0 0x7f6f48139ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 -- 172.21.15.112:0/3540254416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4813b510 con 0x7f6f48150f00 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4cfd0640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481512e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6f4001a4b0 tx=0x7f6f40019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f44ff9640 1 -- 172.21.15.112:0/3540254416 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400b2050 con 0x7f6f48150f00 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f44ff9640 1 -- 172.21.15.112:0/3540254416 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f400be070 con 0x7f6f48150f00 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.695+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4813a210 con 0x7f6f48150f00 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.699+0000 7f6f44ff9640 1 -- 172.21.15.112:0/3540254416 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f400ada50 con 0x7f6f48150f00 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.699+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f48148510 con 0x7f6f48150f00 2024-04-25T00:56:02.700 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.699+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f480750a0 con 0x7f6f48150f00 2024-04-25T00:56:02.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.699+0000 7f6f44ff9640 1 -- 172.21.15.112:0/3540254416 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6f400adbf0 con 0x7f6f48150f00 2024-04-25T00:56:02.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.699+0000 7f6f44ff9640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6f340678c0 0x7f6f34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.699+0000 7f6f44ff9640 1 -- 172.21.15.112:0/3540254416 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6f401218e0 con 0x7f6f48150f00 2024-04-25T00:56:02.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.699+0000 7f6f47fff640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6f340678c0 0x7f6f34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.699+0000 7f6f47fff640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6f340678c0 0x7f6f34069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f38000f70 tx=0x7f6f38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.703+0000 7f6f44ff9640 1 -- 172.21.15.112:0/3540254416 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6f40016500 con 0x7f6f48150f00 2024-04-25T00:56:02.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.851+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f4813a730 con 0x7f6f48150f00 2024-04-25T00:56:02.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.851+0000 7f6f44ff9640 1 -- 172.21.15.112:0/3540254416 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f6f400f2490 con 0x7f6f48150f00 2024-04-25T00:56:02.854 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:56:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6f340678c0 msgr2=0x7f6f34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6f340678c0 0x7f6f34069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6f38000f70 tx=0x7f6f38009040 comp rx=0 tx=0).stop 2024-04-25T00:56:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 msgr2=0x7f6f481512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481512e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6f4001a4b0 tx=0x7f6f40019620 comp rx=0 tx=0).stop 2024-04-25T00:56:02.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 shutdown_connections 2024-04-25T00:56:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f48150f00 0x7f6f481512e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6f480a05a0 0x7f6f48140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6f340678c0 0x7f6f34069d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 --2- 172.21.15.112:0/3540254416 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6f480723f0 0x7f6f48139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 >> 172.21.15.112:0/3540254416 conn(0x7f6f480b75d0 msgr2=0x7f6f480b8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 shutdown_connections 2024-04-25T00:56:02.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.855+0000 7f6f4ea33640 1 -- 172.21.15.112:0/3540254416 wait complete. 2024-04-25T00:56:02.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:56:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f512eba9640 1 Processor -- start 2024-04-25T00:56:02.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f512eba9640 1 -- start start 2024-04-25T00:56:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f512eba9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f51280affb0 0x7f512809da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f512eba9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51280b1390 con 0x7f512809dfa0 2024-04-25T00:56:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f512eba9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51280b1fa0 con 0x7f51280b0390 2024-04-25T00:56:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f512eba9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f51280b1d70 con 0x7f51280affb0 2024-04-25T00:56:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f5127fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f512809dfa0 0x7f5128148800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38708/0 (socket says 172.21.15.112:38708) 2024-04-25T00:56:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f5127fff640 1 -- 172.21.15.112:0/3771467388 learned_addr learned my addr 172.21.15.112:0/3771467388 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f512c945640 1 --2- 172.21.15.112:0/3771467388 >> v2:172.21.15.187:3300/0 conn(0x7f51280affb0 0x7f512809da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.983 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 386267594 0 0) 0x7f51280b1390 con 0x7f512809dfa0 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5110003530 con 0x7f512809dfa0 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 868423574 0 0) 0x7f5110003530 con 0x7f512809dfa0 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 >> v2:172.21.15.187:3300/0 conn(0x7f51280affb0 msgr2=0x7f512809da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 --2- 172.21.15.112:0/3771467388 >> v2:172.21.15.187:3300/0 conn(0x7f51280affb0 0x7f512809da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 >> v1:172.21.15.187:6789/0 conn(0x7f51280b0390 legacy=0x7f512814fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51280b34b0 con 0x7f512809dfa0 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f511c001680 con 0x7f512809dfa0 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 >> v1:172.21.15.112:6789/0 conn(0x7f512809dfa0 legacy=0x7f5128148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51100039c0 0x7f5110003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5110004810 0x7f5110004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51100051d0 0x7f511000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f511000c140 con 0x7f51100039c0 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f511000c2f0 con 0x7f5110004810 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f511000c4f0 con 0x7f51100051d0 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.979+0000 7f512c945640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51100039c0 0x7f5110003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f5127fff640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5110004810 0x7f5110004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512c945640 1 -- 172.21.15.112:0/3771467388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51100051d0 msgr2=0x7f511000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512c945640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51100051d0 0x7f511000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512c945640 1 -- 172.21.15.112:0/3771467388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5110004810 msgr2=0x7f5110004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512c945640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5110004810 0x7f5110004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512d146640 1 -- 172.21.15.112:0/3771467388 reap_dead start 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512c945640 1 -- 172.21.15.112:0/3771467388 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5110003530 con 0x7f51100039c0 2024-04-25T00:56:02.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f5127fff640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5110004810 0x7f5110004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:02.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512c945640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51100039c0 0x7f5110003de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5120009fc0 tx=0x7f512001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da12e640b5743c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51200b2070 con 0x7f51100039c0 2024-04-25T00:56:02.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51200a5a90 con 0x7f51100039c0 2024-04-25T00:56:02.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f51277fe640 1 -- 172.21.15.112:0/3771467388 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120009c10 con 0x7f51100039c0 2024-04-25T00:56:02.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 -- 172.21.15.112:0/3771467388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51100039c0 msgr2=0x7f5110003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51100039c0 0x7f5110003de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5120009fc0 tx=0x7f512001cca0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 -- 172.21.15.112:0/3771467388 shutdown_connections 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 --2- 172.21.15.112:0/3771467388 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f51100039c0 0x7f5110003de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 -- 172.21.15.112:0/3771467388 >> 172.21.15.112:0/3771467388 conn(0x7f51280b8c80 msgr2=0x7f51280b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 -- 172.21.15.112:0/3771467388 shutdown_connections 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 -- 172.21.15.112:0/3771467388 wait complete. 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 Processor -- start 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.983+0000 7f512eba9640 1 -- start start 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51100039c0 0x7f51281e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5128070b70 0x7f51281e2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51281e28d0 0x7f512809ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5128145190 con 0x7f5128070b70 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5128149220 con 0x7f51281e28d0 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5128145310 con 0x7f51100039c0 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5128070b70 0x7f51281e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5128070b70 0x7f51281e2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60506/0 (socket says 172.21.15.112:60506) 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 -- 172.21.15.112:0/70248135 learned_addr learned my addr 172.21.15.112:0/70248135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512c945640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51100039c0 0x7f51281e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 -- 172.21.15.112:0/70248135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51100039c0 msgr2=0x7f51281e1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51100039c0 0x7f51281e1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 -- 172.21.15.112:0/70248135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51281e28d0 msgr2=0x7f512809ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51281e28d0 0x7f512809ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 -- 172.21.15.112:0/70248135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5128071080 con 0x7f5128070b70 2024-04-25T00:56:02.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f5127fff640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5128070b70 0x7f51281e2390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f511c00d810 tx=0x7f511c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f51257fa640 1 -- 172.21.15.112:0/70248135 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c019070 con 0x7f5128070b70 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f51257fa640 1 -- 172.21.15.112:0/70248135 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c0074e0 con 0x7f5128070b70 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5128071360 con 0x7f5128070b70 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f51257fa640 1 -- 172.21.15.112:0/70248135 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0049f0 con 0x7f5128070b70 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5128070640 con 0x7f5128070b70 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5128071580 con 0x7f5128070b70 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f51257fa640 1 -- 172.21.15.112:0/70248135 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f511c01c070 con 0x7f5128070b70 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f51257fa640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f51140678c0 0x7f5114069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f51257fa640 1 -- 172.21.15.112:0/70248135 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f511c0857e0 con 0x7f5128070b70 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512c945640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f51140678c0 0x7f5114069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:02.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.987+0000 7f512c945640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f51140678c0 0x7f5114069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f51200af000 tx=0x7f51200a7040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:02.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:02.991+0000 7f51257fa640 1 -- 172.21.15.112:0/70248135 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f511c056310 con 0x7f5128070b70 2024-04-25T00:56:03.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.139+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f51280b8630 con 0x7f5128070b70 2024-04-25T00:56:03.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.139+0000 7f51257fa640 1 -- 172.21.15.112:0/70248135 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f511c055c10 con 0x7f5128070b70 2024-04-25T00:56:03.142 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:56:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f51140678c0 msgr2=0x7f5114069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f51140678c0 0x7f5114069d80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f51200af000 tx=0x7f51200a7040 comp rx=0 tx=0).stop 2024-04-25T00:56:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5128070b70 msgr2=0x7f51281e2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5128070b70 0x7f51281e2390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f511c00d810 tx=0x7f511c00dcd0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 shutdown_connections 2024-04-25T00:56:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f51140678c0 0x7f5114069d80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f51281e28d0 0x7f512809ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5128070b70 0x7f51281e2390 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 --2- 172.21.15.112:0/70248135 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f51100039c0 0x7f51281e1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 >> 172.21.15.112:0/70248135 conn(0x7f51280b8c80 msgr2=0x7f5128143100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 shutdown_connections 2024-04-25T00:56:03.145 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.143+0000 7f512eba9640 1 -- 172.21.15.112:0/70248135 wait complete. 2024-04-25T00:56:03.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:56:03.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.279+0000 7f7ac2878640 1 Processor -- start 2024-04-25T00:56:03.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.279+0000 7f7ac2878640 1 -- start start 2024-04-25T00:56:03.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac2878640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7abc0b0390 0x7f7abc148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac2878640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7abc072b20 0x7f7abc148d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac2878640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7abc0b1390 con 0x7f7abc0affb0 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac2878640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7abc0b1d70 con 0x7f7abc072b20 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac2878640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f7abc0b1600 con 0x7f7abc0b0390 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac0e15640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7abc072b20 0x7f7abc148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac0e15640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7abc072b20 0x7f7abc148d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60518/0 (socket says 172.21.15.112:60518) 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac0e15640 1 -- 172.21.15.112:0/4289673645 learned_addr learned my addr 172.21.15.112:0/4289673645 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ab3fff640 1 --2- 172.21.15.112:0/4289673645 >> v2:172.21.15.187:3300/0 conn(0x7f7abc0b0390 0x7f7abc148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac0e15640 1 -- 172.21.15.112:0/4289673645 >> v2:172.21.15.187:3300/0 conn(0x7f7abc0b0390 msgr2=0x7f7abc148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac0e15640 1 --2- 172.21.15.112:0/4289673645 >> v2:172.21.15.187:3300/0 conn(0x7f7abc0b0390 0x7f7abc148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac0e15640 1 -- 172.21.15.112:0/4289673645 >> v1:172.21.15.187:6789/0 conn(0x7f7abc0affb0 legacy=0x7f7abc09da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac0e15640 1 -- 172.21.15.112:0/4289673645 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc0b1fa0 con 0x7f7abc072b20 2024-04-25T00:56:03.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ab3fff640 1 --2- 172.21.15.112:0/4289673645 >> v2:172.21.15.187:3300/0 conn(0x7f7abc0b0390 0x7f7abc148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:03.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7ac0e15640 1 --2- 172.21.15.112:0/4289673645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc148d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0b07c0 tx=0x7f7ab401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5791e6bcf978b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7abb7fe640 1 -- 172.21.15.112:0/4289673645 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40a57e0 con 0x7f7abc072b20 2024-04-25T00:56:03.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7abb7fe640 1 -- 172.21.15.112:0/4289673645 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab40a5980 con 0x7f7abc072b20 2024-04-25T00:56:03.285 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.283+0000 7f7abb7fe640 1 -- 172.21.15.112:0/4289673645 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40a5d50 con 0x7f7abc072b20 2024-04-25T00:56:03.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- 172.21.15.112:0/4289673645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 msgr2=0x7f7abc148d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 --2- 172.21.15.112:0/4289673645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc148d40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7abc0b07c0 tx=0x7f7ab401c920 comp rx=0 tx=0).stop 2024-04-25T00:56:03.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- 172.21.15.112:0/4289673645 shutdown_connections 2024-04-25T00:56:03.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 --2- 172.21.15.112:0/4289673645 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc148d40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 --2- 172.21.15.112:0/4289673645 >> v2:172.21.15.187:3300/0 conn(0x7f7abc0b0390 0x7f7abc148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- 172.21.15.112:0/4289673645 >> 172.21.15.112:0/4289673645 conn(0x7f7abc0b8c80 msgr2=0x7f7abc0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:03.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- 172.21.15.112:0/4289673645 shutdown_connections 2024-04-25T00:56:03.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- 172.21.15.112:0/4289673645 wait complete. 2024-04-25T00:56:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 Processor -- start 2024-04-25T00:56:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- start start 2024-04-25T00:56:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc0af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7abc0affb0 0x7f7abc0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7abc0b0390 0x7f7abc1dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc152290 con 0x7f7abc072b20 2024-04-25T00:56:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7abc09e170 con 0x7f7abc0affb0 2024-04-25T00:56:03.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7abc152410 con 0x7f7abc0b0390 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc0af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc0af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60522/0 (socket says 172.21.15.112:60522) 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 -- 172.21.15.112:0/547461759 learned_addr learned my addr 172.21.15.112:0/547461759 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac0e15640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7abc0b0390 0x7f7abc1dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ab3fff640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7abc0affb0 0x7f7abc0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 -- 172.21.15.112:0/547461759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7abc0b0390 msgr2=0x7f7abc1dfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7abc0b0390 0x7f7abc1dfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 -- 172.21.15.112:0/547461759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7abc0affb0 msgr2=0x7f7abc0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7abc0affb0 0x7f7abc0ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 -- 172.21.15.112:0/547461759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7abc09b240 con 0x7f7abc072b20 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7abbfff640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc0af670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c450 tx=0x7f7aa400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ab97fa640 1 -- 172.21.15.112:0/547461759 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4012830 con 0x7f7abc072b20 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ab97fa640 1 -- 172.21.15.112:0/547461759 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa4017070 con 0x7f7abc072b20 2024-04-25T00:56:03.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7abc09ac80 con 0x7f7abc072b20 2024-04-25T00:56:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ab97fa640 1 -- 172.21.15.112:0/547461759 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4012c30 con 0x7f7abc072b20 2024-04-25T00:56:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7abc09c390 con 0x7f7abc072b20 2024-04-25T00:56:03.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.287+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7abc0b1040 con 0x7f7abc072b20 2024-04-25T00:56:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.291+0000 7f7ab97fa640 1 -- 172.21.15.112:0/547461759 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7aa401c460 con 0x7f7abc072b20 2024-04-25T00:56:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.291+0000 7f7ab97fa640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7a9c0678c0 0x7f7a9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.291+0000 7f7ab97fa640 1 -- 172.21.15.112:0/547461759 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7aa40859b0 con 0x7f7abc072b20 2024-04-25T00:56:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.291+0000 7f7ab3fff640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7a9c0678c0 0x7f7a9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.291+0000 7f7ab3fff640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7a9c0678c0 0x7f7a9c069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7aac006fd0 tx=0x7f7aac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.295+0000 7f7ab97fa640 1 -- 172.21.15.112:0/547461759 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7aa4056580 con 0x7f7abc072b20 2024-04-25T00:56:03.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7abc04f5f0 con 0x7f7abc072b20 2024-04-25T00:56:03.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ab97fa640 1 -- 172.21.15.112:0/547461759 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f7aa4055e80 con 0x7f7abc072b20 2024-04-25T00:56:03.456 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:56:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7a9c0678c0 msgr2=0x7f7a9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7a9c0678c0 0x7f7a9c069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7aac006fd0 tx=0x7f7aac008040 comp rx=0 tx=0).stop 2024-04-25T00:56:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 msgr2=0x7f7abc0af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc0af670 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400c450 tx=0x7f7aa400c910 comp rx=0 tx=0).stop 2024-04-25T00:56:03.458 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 shutdown_connections 2024-04-25T00:56:03.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7abc0b0390 0x7f7abc1dfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7a9c0678c0 0x7f7a9c069d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7abc0affb0 0x7f7abc0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 --2- 172.21.15.112:0/547461759 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7abc072b20 0x7f7abc0af670 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 >> 172.21.15.112:0/547461759 conn(0x7f7abc0b8c80 msgr2=0x7f7abc0b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:03.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.455+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 shutdown_connections 2024-04-25T00:56:03.459 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.459+0000 7f7ac2878640 1 -- 172.21.15.112:0/547461759 wait complete. 2024-04-25T00:56:03.475 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:56:03.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f762d8d4640 1 Processor -- start 2024-04-25T00:56:03.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f762d8d4640 1 -- start start 2024-04-25T00:56:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f762d8d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 0x7f76280727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f762d8d4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76280b5300 con 0x7f76280b47d0 2024-04-25T00:56:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f762d8d4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f762806c900 con 0x7f7628141820 2024-04-25T00:56:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f762d8d4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f76280ae670 con 0x7f762806c520 2024-04-25T00:56:03.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76267fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f76280b47d0 0x7f7628141110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38718/0 (socket says 172.21.15.112:38718) 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76267fc640 1 -- 172.21.15.112:0/1828624 learned_addr learned my addr 172.21.15.112:0/1828624 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7626ffd640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 0x7f76280727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2451192099 0 0) 0x7f76280b5300 con 0x7f76280b47d0 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7614003530 con 0x7f76280b47d0 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2026362988 0 0) 0x7f7614003530 con 0x7f76280b47d0 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 msgr2=0x7f76280727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 0x7f76280727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 >> v1:172.21.15.187:6789/0 conn(0x7f7628141820 legacy=0x7f7628151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76280ae950 con 0x7f76280b47d0 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f7610001680 con 0x7f76280b47d0 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 >> v1:172.21.15.112:6789/0 conn(0x7f76280b47d0 legacy=0x7f7628141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76140039f0 0x7f7614003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7614004810 0x7f7614004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 0x7f7614005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76280ae670 con 0x7f7614004810 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f761400bb00 con 0x7f76140039f0 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f761400bd00 con 0x7f762806c520 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76267fc640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7614004810 0x7f7614004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76277fe640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 0x7f7614005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f7626ffd640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76140039f0 0x7f7614003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76267fc640 1 -- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 msgr2=0x7f7614005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76267fc640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 0x7f7614005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76267fc640 1 -- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76140039f0 msgr2=0x7f7614003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76267fc640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76140039f0 0x7f7614003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76267fc640 1 -- 172.21.15.112:0/1828624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7614003530 con 0x7f7614004810 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.595+0000 7f76277fe640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f762806c520 0x7f7614005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f76267fc640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7614004810 0x7f7614004c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f761000c240 tx=0x7f761001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550e3801dcc6946c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610007d20 con 0x7f7614004810 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76100a9070 con 0x7f7614004810 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f7625ffb640 1 -- 172.21.15.112:0/1828624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610005d40 con 0x7f7614004810 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- 172.21.15.112:0/1828624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7614004810 msgr2=0x7f7614004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 --2- 172.21.15.112:0/1828624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7614004810 0x7f7614004c90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f761000c240 tx=0x7f761001cc80 comp rx=0 tx=0).stop 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f76277fe640 1 -- 172.21.15.112:0/1828624 reap_dead start 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- 172.21.15.112:0/1828624 shutdown_connections 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- 172.21.15.112:0/1828624 >> 172.21.15.112:0/1828624 conn(0x7f76280b5980 msgr2=0x7f76280b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:03.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- 172.21.15.112:0/1828624 shutdown_connections 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- 172.21.15.112:0/1828624 wait complete. 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 Processor -- start 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- start start 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7614004810 0x7f76281e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7628144210 0x7f762814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762814b4f0 0x7f7628148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7628153640 con 0x7f7628144210 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7628141c30 con 0x7f762814b4f0 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f762d8d4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f76281537c0 con 0x7f7614004810 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f76267fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7628144210 0x7f762814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f76267fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7628144210 0x7f762814afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60538/0 (socket says 172.21.15.112:60538) 2024-04-25T00:56:03.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.599+0000 7f76267fc640 1 -- 172.21.15.112:0/2681491879 learned_addr learned my addr 172.21.15.112:0/2681491879 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f76277fe640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762814b4f0 0x7f7628148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f7626ffd640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7614004810 0x7f76281e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f76267fc640 1 -- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7614004810 msgr2=0x7f76281e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f76267fc640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7614004810 0x7f76281e08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f76267fc640 1 -- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762814b4f0 msgr2=0x7f7628148dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f76267fc640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762814b4f0 0x7f7628148dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f76267fc640 1 -- 172.21.15.112:0/2681491879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762814a5e0 con 0x7f7628144210 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f76277fe640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762814b4f0 0x7f7628148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f7626ffd640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7614004810 0x7f76281e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f76267fc640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7628144210 0x7f762814afb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f761001cab0 tx=0x7f7614005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f761ffff640 1 -- 172.21.15.112:0/2681491879 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610007d20 con 0x7f7628144210 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f761ffff640 1 -- 172.21.15.112:0/2681491879 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76100047f0 con 0x7f7628144210 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76281e00d0 con 0x7f7628144210 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f761ffff640 1 -- 172.21.15.112:0/2681491879 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7610004ba0 con 0x7f7628144210 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762814a850 con 0x7f7628144210 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7628144700 con 0x7f7628144210 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f761ffff640 1 -- 172.21.15.112:0/2681491879 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7610007070 con 0x7f7628144210 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f761ffff640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76080678c0 0x7f7608069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f7626ffd640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76080678c0 0x7f7608069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f761ffff640 1 -- 172.21.15.112:0/2681491879 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f761011f3b0 con 0x7f7628144210 2024-04-25T00:56:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.603+0000 7f7626ffd640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76080678c0 0x7f7608069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7620007460 tx=0x7f762000d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.607+0000 7f761ffff640 1 -- 172.21.15.112:0/2681491879 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7610016990 con 0x7f7628144210 2024-04-25T00:56:03.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.771+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f762804f580 con 0x7f7628144210 2024-04-25T00:56:03.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.771+0000 7f761ffff640 1 -- 172.21.15.112:0/2681491879 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f76100efef0 con 0x7f7628144210 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76080678c0 msgr2=0x7f7608069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76080678c0 0x7f7608069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7620007460 tx=0x7f762000d040 comp rx=0 tx=0).stop 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7628144210 msgr2=0x7f762814afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7628144210 0x7f762814afb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f761001cab0 tx=0x7f7614005840 comp rx=0 tx=0).stop 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 shutdown_connections 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f76080678c0 0x7f7608069d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f762814b4f0 0x7f7628148dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7628144210 0x7f762814afb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 --2- 172.21.15.112:0/2681491879 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7614004810 0x7f76281e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 >> 172.21.15.112:0/2681491879 conn(0x7f76280b5980 msgr2=0x7f76280b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 shutdown_connections 2024-04-25T00:56:03.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.775+0000 7f762d8d4640 1 -- 172.21.15.112:0/2681491879 wait complete. 2024-04-25T00:56:03.790 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:56:03.790 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:56:03.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f80014a2640 1 Processor -- start 2024-04-25T00:56:03.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f80014a2640 1 -- start start 2024-04-25T00:56:03.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f80014a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0add70 0x7f7ffc145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f80014a2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ffc0b3ce0 0x7f7ffc14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f80014a2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ffc0aeec0 con 0x7f7ffc0b23f0 2024-04-25T00:56:03.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f80014a2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7ffc0af780 con 0x7f7ffc0b3ce0 2024-04-25T00:56:03.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f80014a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc0b4170 con 0x7f7ffc0add70 2024-04-25T00:56:03.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffaffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7ffc0b23f0 0x7f7ffc072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38732/0 (socket says 172.21.15.112:38732) 2024-04-25T00:56:03.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffaffd640 1 -- 172.21.15.112:0/3437864027 learned_addr learned my addr 172.21.15.112:0/3437864027 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:03.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffb7fe640 1 --2- 172.21.15.112:0/3437864027 >> v2:172.21.15.112:3300/0 conn(0x7f7ffc0b3ce0 0x7f7ffc14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ff27fc640 1 --2- 172.21.15.112:0/3437864027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0add70 0x7f7ffc145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffa7fc640 1 -- 172.21.15.112:0/3437864027 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 219016233 0 0) 0x7f7ffc0aeec0 con 0x7f7ffc0b23f0 2024-04-25T00:56:03.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffa7fc640 1 -- 172.21.15.112:0/3437864027 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe8003530 con 0x7f7ffc0b23f0 2024-04-25T00:56:03.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffb7fe640 1 -- 172.21.15.112:0/3437864027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0add70 msgr2=0x7f7ffc145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:03.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffb7fe640 1 --2- 172.21.15.112:0/3437864027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0add70 0x7f7ffc145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffb7fe640 1 -- 172.21.15.112:0/3437864027 >> v1:172.21.15.112:6789/0 conn(0x7f7ffc0b23f0 legacy=0x7f7ffc072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffb7fe640 1 -- 172.21.15.112:0/3437864027 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc0b27d0 con 0x7f7ffc0b3ce0 2024-04-25T00:56:03.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffb7fe640 1 --2- 172.21.15.112:0/3437864027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0b3ce0 0x7f7ffc14fb40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009d90 tx=0x7f7ff401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7ba04f3ec8da9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffa7fc640 1 -- 172.21.15.112:0/3437864027 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40b2070 con 0x7f7ffc0b3ce0 2024-04-25T00:56:03.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffa7fc640 1 -- 172.21.15.112:0/3437864027 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff40a5940 con 0x7f7ffc0b3ce0 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.951+0000 7f7ffa7fc640 1 -- 172.21.15.112:0/3437864027 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40a5cf0 con 0x7f7ffc0b3ce0 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- 172.21.15.112:0/3437864027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0b3ce0 msgr2=0x7f7ffc14fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 --2- 172.21.15.112:0/3437864027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0b3ce0 0x7f7ffc14fb40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009d90 tx=0x7f7ff401c970 comp rx=0 tx=0).stop 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- 172.21.15.112:0/3437864027 shutdown_connections 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 --2- 172.21.15.112:0/3437864027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0b3ce0 0x7f7ffc14fb40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 --2- 172.21.15.112:0/3437864027 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0add70 0x7f7ffc145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- 172.21.15.112:0/3437864027 >> 172.21.15.112:0/3437864027 conn(0x7f7ffc0b5980 msgr2=0x7f7ffc0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- 172.21.15.112:0/3437864027 shutdown_connections 2024-04-25T00:56:03.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- 172.21.15.112:0/3437864027 wait complete. 2024-04-25T00:56:03.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 Processor -- start 2024-04-25T00:56:03.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- start start 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0add70 0x7f7ffc139330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ffc0b23f0 0x7f7ffc1e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0b3ce0 0x7f7ffc1e1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc0b52d0 con 0x7f7ffc0add70 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ffc0b5150 con 0x7f7ffc0b23f0 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ffc1528a0 con 0x7f7ffc0b3ce0 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0add70 0x7f7ffc139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0add70 0x7f7ffc139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60560/0 (socket says 172.21.15.112:60560) 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 -- 172.21.15.112:0/966302314 learned_addr learned my addr 172.21.15.112:0/966302314 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 -- 172.21.15.112:0/966302314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0b3ce0 msgr2=0x7f7ffc1e1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0b3ce0 0x7f7ffc1e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 -- 172.21.15.112:0/966302314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ffc0b23f0 msgr2=0x7f7ffc1e13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ffc0b23f0 0x7f7ffc1e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 -- 172.21.15.112:0/966302314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ffc142e80 con 0x7f7ffc0add70 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ffaffd640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0add70 0x7f7ffc139330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00c450 tx=0x7f7fec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ff3fff640 1 -- 172.21.15.112:0/966302314 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec017070 con 0x7f7ffc0add70 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ff3fff640 1 -- 172.21.15.112:0/966302314 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fec005db0 con 0x7f7ffc0add70 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ffc06f1d0 con 0x7f7ffc0add70 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f7ff3fff640 1 -- 172.21.15.112:0/966302314 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fec0072f0 con 0x7f7ffc0add70 2024-04-25T00:56:03.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ffc13e780 con 0x7f7ffc0add70 2024-04-25T00:56:03.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.955+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ffc04f580 con 0x7f7ffc0add70 2024-04-25T00:56:03.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.959+0000 7f7ff3fff640 1 -- 172.21.15.112:0/966302314 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7fec01a070 con 0x7f7ffc0add70 2024-04-25T00:56:03.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.959+0000 7f7ff3fff640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7fdc0678c0 0x7f7fdc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:03.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.959+0000 7f7ff27fc640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7fdc0678c0 0x7f7fdc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:03.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.959+0000 7f7ff3fff640 1 -- 172.21.15.112:0/966302314 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7fec0850c0 con 0x7f7ffc0add70 2024-04-25T00:56:03.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.959+0000 7f7ff27fc640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7fdc0678c0 0x7f7fdc069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc071c00 tx=0x7f7fe4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:03.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:03.963+0000 7f7ff3fff640 1 -- 172.21.15.112:0/966302314 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7fec055c90 con 0x7f7ffc0add70 2024-04-25T00:56:04.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.111+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ffc0b8290 con 0x7f7ffc0add70 2024-04-25T00:56:04.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.111+0000 7f7ff3fff640 1 -- 172.21.15.112:0/966302314 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f7fec055590 con 0x7f7ffc0add70 2024-04-25T00:56:04.114 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:04.114 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7fdc0678c0 msgr2=0x7f7fdc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7fdc0678c0 0x7f7fdc069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc071c00 tx=0x7f7fe4008040 comp rx=0 tx=0).stop 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0add70 msgr2=0x7f7ffc139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0add70 0x7f7ffc139330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fec00c450 tx=0x7f7fec00c910 comp rx=0 tx=0).stop 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 shutdown_connections 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ffc0b3ce0 0x7f7ffc1e1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7fdc0678c0 0x7f7fdc069d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ffc0b23f0 0x7f7ffc1e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.116 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 --2- 172.21.15.112:0/966302314 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ffc0add70 0x7f7ffc139330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 >> 172.21.15.112:0/966302314 conn(0x7f7ffc0b5980 msgr2=0x7f7ffc0b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:04.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 shutdown_connections 2024-04-25T00:56:04.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.115+0000 7f80014a2640 1 -- 172.21.15.112:0/966302314 wait complete. 2024-04-25T00:56:04.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:56:04.132 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:56:04.257 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.255+0000 7fb05227e640 1 Processor -- start 2024-04-25T00:56:04.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.255+0000 7fb05227e640 1 -- start start 2024-04-25T00:56:04.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb04c148f10 0x7fb04c150a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04c0b1390 con 0x7fb04c0affb0 2024-04-25T00:56:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb04c0b1fa0 con 0x7fb04c0b0390 2024-04-25T00:56:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb04c0b1d70 con 0x7fb04c148f10 2024-04-25T00:56:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb04bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb04c0affb0 0x7fb04c09da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38748/0 (socket says 172.21.15.112:38748) 2024-04-25T00:56:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb04bfff640 1 -- 172.21.15.112:0/2236784578 learned_addr learned my addr 172.21.15.112:0/2236784578 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:04.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05081b640 1 --2- 172.21.15.112:0/2236784578 >> v2:172.21.15.112:3300/0 conn(0x7fb04c148f10 0x7fb04c150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb04affd640 1 -- 172.21.15.112:0/2236784578 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3858972838 0 0) 0x7fb04c0b1390 con 0x7fb04c0affb0 2024-04-25T00:56:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb04affd640 1 -- 172.21.15.112:0/2236784578 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb034003530 con 0x7fb04c0affb0 2024-04-25T00:56:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05081b640 1 -- 172.21.15.112:0/2236784578 >> v1:172.21.15.187:6789/0 conn(0x7fb04c0b0390 legacy=0x7fb04c148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05081b640 1 -- 172.21.15.112:0/2236784578 >> v1:172.21.15.112:6789/0 conn(0x7fb04c0affb0 legacy=0x7fb04c09da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05081b640 1 -- 172.21.15.112:0/2236784578 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04c0b34b0 con 0x7fb04c148f10 2024-04-25T00:56:04.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05081b640 1 --2- 172.21.15.112:0/2236784578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c150a80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb044009d90 tx=0x7fb04401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80fcffc4c031e7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb04affd640 1 -- 172.21.15.112:0/2236784578 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440b2070 con 0x7fb04c148f10 2024-04-25T00:56:04.268 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb04affd640 1 -- 172.21.15.112:0/2236784578 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0440a5890 con 0x7fb04c148f10 2024-04-25T00:56:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb04affd640 1 -- 172.21.15.112:0/2236784578 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440a5c60 con 0x7fb04c148f10 2024-04-25T00:56:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 -- 172.21.15.112:0/2236784578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 msgr2=0x7fb04c150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 --2- 172.21.15.112:0/2236784578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c150a80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb044009d90 tx=0x7fb04401c8e0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 -- 172.21.15.112:0/2236784578 shutdown_connections 2024-04-25T00:56:04.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 --2- 172.21.15.112:0/2236784578 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c150a80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 -- 172.21.15.112:0/2236784578 >> 172.21.15.112:0/2236784578 conn(0x7fb04c0b8c80 msgr2=0x7fb04c0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 -- 172.21.15.112:0/2236784578 shutdown_connections 2024-04-25T00:56:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.259+0000 7fb05227e640 1 -- 172.21.15.112:0/2236784578 wait complete. 2024-04-25T00:56:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 Processor -- start 2024-04-25T00:56:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 -- start start 2024-04-25T00:56:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb04c0affb0 0x7fb04c144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb04c0b0390 0x7fb04c144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c153960 con 0x7fb04c148f10 2024-04-25T00:56:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb04c1537e0 con 0x7fb04c0b0390 2024-04-25T00:56:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb04c153ae0 con 0x7fb04c0affb0 2024-04-25T00:56:04.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60580/0 (socket says 172.21.15.112:60580) 2024-04-25T00:56:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 -- 172.21.15.112:0/518982948 learned_addr learned my addr 172.21.15.112:0/518982948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb04bfff640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb04c0affb0 0x7fb04c144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb04b7fe640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb04c0b0390 0x7fb04c144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 -- 172.21.15.112:0/518982948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb04c0affb0 msgr2=0x7fb04c144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb04c0affb0 0x7fb04c144050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 -- 172.21.15.112:0/518982948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb04c0b0390 msgr2=0x7fb04c144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb04c0b0390 0x7fb04c144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 -- 172.21.15.112:0/518982948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04c145300 con 0x7fb04c148f10 2024-04-25T00:56:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05081b640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c141f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0440b0040 tx=0x7fb044019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb048ff9640 1 -- 172.21.15.112:0/518982948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440b2050 con 0x7fb04c148f10 2024-04-25T00:56:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb048ff9640 1 -- 172.21.15.112:0/518982948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0440be070 con 0x7fb04c148f10 2024-04-25T00:56:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04c1455e0 con 0x7fb04c148f10 2024-04-25T00:56:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb048ff9640 1 -- 172.21.15.112:0/518982948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440ada70 con 0x7fb04c148f10 2024-04-25T00:56:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04c144ff0 con 0x7fb04c148f10 2024-04-25T00:56:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04c0b1040 con 0x7fb04c148f10 2024-04-25T00:56:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.263+0000 7fb048ff9640 1 -- 172.21.15.112:0/518982948 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb0440a5710 con 0x7fb04c148f10 2024-04-25T00:56:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.267+0000 7fb048ff9640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb0380678f0 0x7fb038069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.267+0000 7fb048ff9640 1 -- 172.21.15.112:0/518982948 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb044120d00 con 0x7fb04c148f10 2024-04-25T00:56:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.267+0000 7fb04bfff640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb0380678f0 0x7fb038069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.267+0000 7fb04bfff640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb0380678f0 0x7fb038069db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0011f0 tx=0x7fb03c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.271+0000 7fb048ff9640 1 -- 172.21.15.112:0/518982948 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb044016560 con 0x7fb04c148f10 2024-04-25T00:56:04.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb04c04f5f0 con 0x7fb04c148f10 2024-04-25T00:56:04.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb048ff9640 1 -- 172.21.15.112:0/518982948 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fb0440f18d0 con 0x7fb04c148f10 2024-04-25T00:56:04.420 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb0380678f0 msgr2=0x7fb038069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb0380678f0 0x7fb038069db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0011f0 tx=0x7fb03c009040 comp rx=0 tx=0).stop 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 msgr2=0x7fb04c141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c141f60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb0440b0040 tx=0x7fb044019680 comp rx=0 tx=0).stop 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 shutdown_connections 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb04c148f10 0x7fb04c141f60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb0380678f0 0x7fb038069db0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb04c0b0390 0x7fb04c144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 --2- 172.21.15.112:0/518982948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb04c0affb0 0x7fb04c144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 >> 172.21.15.112:0/518982948 conn(0x7fb04c0b8c80 msgr2=0x7fb04c0b3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 shutdown_connections 2024-04-25T00:56:04.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.419+0000 7fb05227e640 1 -- 172.21.15.112:0/518982948 wait complete. 2024-04-25T00:56:04.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:56:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.547+0000 7f3bfc7e0640 1 Processor -- start 2024-04-25T00:56:04.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.547+0000 7f3bfc7e0640 1 -- start start 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40affb0 0x7f3bf409da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3bf40b0390 0x7f3bf414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bf40b1390 con 0x7f3bf409dfa0 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3bf40b1d70 con 0x7f3bf40b0390 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf40b1600 con 0x7f3bf40affb0 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfad7d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3bf40b0390 0x7f3bf414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bf9d7b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3bf409dfa0 0x7f3bf4148800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38750/0 (socket says 172.21.15.112:38750) 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bf9d7b640 1 -- 172.21.15.112:0/1963370488 learned_addr learned my addr 172.21.15.112:0/1963370488 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfa57c640 1 --2- 172.21.15.112:0/1963370488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40affb0 0x7f3bf409da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfad7d640 1 -- 172.21.15.112:0/1963370488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40affb0 msgr2=0x7f3bf409da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:04.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfad7d640 1 --2- 172.21.15.112:0/1963370488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40affb0 0x7f3bf409da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bf957a640 1 -- 172.21.15.112:0/1963370488 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648748317 0 0) 0x7f3bf40b1390 con 0x7f3bf409dfa0 2024-04-25T00:56:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfad7d640 1 -- 172.21.15.112:0/1963370488 >> v1:172.21.15.112:6789/0 conn(0x7f3bf409dfa0 legacy=0x7f3bf4148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfad7d640 1 -- 172.21.15.112:0/1963370488 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf40b1fa0 con 0x7f3bf40b0390 2024-04-25T00:56:04.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfad7d640 1 --2- 172.21.15.112:0/1963370488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40b0390 0x7f3bf414fa70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009dc0 tx=0x7f3bec01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6bac193c322f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bf957a640 1 -- 172.21.15.112:0/1963370488 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0b2070 con 0x7f3bf40b0390 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bf957a640 1 -- 172.21.15.112:0/1963370488 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec0a5980 con 0x7f3bf40b0390 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bf957a640 1 -- 172.21.15.112:0/1963370488 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0a5d50 con 0x7f3bf40b0390 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/1963370488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40b0390 msgr2=0x7f3bf414fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/1963370488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40b0390 0x7f3bf414fa70 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009dc0 tx=0x7f3bec01c8c0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/1963370488 shutdown_connections 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/1963370488 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf40b0390 0x7f3bf414fa70 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/1963370488 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40affb0 0x7f3bf409da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/1963370488 >> 172.21.15.112:0/1963370488 conn(0x7f3bf40b8c80 msgr2=0x7f3bf40b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:04.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/1963370488 shutdown_connections 2024-04-25T00:56:04.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.551+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/1963370488 wait complete. 2024-04-25T00:56:04.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 Processor -- start 2024-04-25T00:56:04.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 -- start start 2024-04-25T00:56:04.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf409dfa0 0x7f3bf41467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40affb0 0x7f3bf4138450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40b0390 0x7f3bf409aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf4152950 con 0x7f3bf409dfa0 2024-04-25T00:56:04.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf41527d0 con 0x7f3bf40affb0 2024-04-25T00:56:04.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3bf4152ad0 con 0x7f3bf40b0390 2024-04-25T00:56:04.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf409dfa0 0x7f3bf41467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf409dfa0 0x7f3bf41467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60598/0 (socket says 172.21.15.112:60598) 2024-04-25T00:56:04.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 -- 172.21.15.112:0/3950038025 learned_addr learned my addr 172.21.15.112:0/3950038025 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:04.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfad7d640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40b0390 0x7f3bf409aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bf9d7b640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40affb0 0x7f3bf4138450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 -- 172.21.15.112:0/3950038025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40b0390 msgr2=0x7f3bf409aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40b0390 0x7f3bf409aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 -- 172.21.15.112:0/3950038025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40affb0 msgr2=0x7f3bf4138450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40affb0 0x7f3bf4138450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 -- 172.21.15.112:0/3950038025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf4147d50 con 0x7f3bf409dfa0 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfa57c640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf409dfa0 0x7f3bf41467e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3be400d800 tx=0x7f3be400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3beaffd640 1 -- 172.21.15.112:0/3950038025 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be400b660 con 0x7f3bf409dfa0 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3beaffd640 1 -- 172.21.15.112:0/3950038025 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3be4019070 con 0x7f3bf409dfa0 2024-04-25T00:56:04.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf409cee0 con 0x7f3bf409dfa0 2024-04-25T00:56:04.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3beaffd640 1 -- 172.21.15.112:0/3950038025 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3be400ba80 con 0x7f3bf409dfa0 2024-04-25T00:56:04.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf409d4a0 con 0x7f3bf409dfa0 2024-04-25T00:56:04.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bf4072640 con 0x7f3bf409dfa0 2024-04-25T00:56:04.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3beaffd640 1 -- 172.21.15.112:0/3950038025 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3be400bc20 con 0x7f3bf409dfa0 2024-04-25T00:56:04.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3beaffd640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3bd80678c0 0x7f3bd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3bf9d7b640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3bd80678c0 0x7f3bd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.555+0000 7f3beaffd640 1 -- 172.21.15.112:0/3950038025 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3be40860e0 con 0x7f3bf409dfa0 2024-04-25T00:56:04.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.559+0000 7f3bf9d7b640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3bd80678c0 0x7f3bd8069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40725d0 tx=0x7f3bf0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.563+0000 7f3beaffd640 1 -- 172.21.15.112:0/3950038025 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3be4056cb0 con 0x7f3bf409dfa0 2024-04-25T00:56:04.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.707+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3bf404f5f0 con 0x7f3bf409dfa0 2024-04-25T00:56:04.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.707+0000 7f3beaffd640 1 -- 172.21.15.112:0/3950038025 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f3be40565b0 con 0x7f3bf409dfa0 2024-04-25T00:56:04.709 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:56:04.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3bd80678c0 msgr2=0x7f3bd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3bd80678c0 0x7f3bd8069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3bf40725d0 tx=0x7f3bf0009040 comp rx=0 tx=0).stop 2024-04-25T00:56:04.711 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf409dfa0 msgr2=0x7f3bf41467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf409dfa0 0x7f3bf41467e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3be400d800 tx=0x7f3be400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 shutdown_connections 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3bf40b0390 0x7f3bf409aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3bd80678c0 0x7f3bd8069d80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3bf40affb0 0x7f3bf4138450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 --2- 172.21.15.112:0/3950038025 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3bf409dfa0 0x7f3bf41467e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 >> 172.21.15.112:0/3950038025 conn(0x7f3bf40b8c80 msgr2=0x7f3bf40b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 shutdown_connections 2024-04-25T00:56:04.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.711+0000 7f3bfc7e0640 1 -- 172.21.15.112:0/3950038025 wait complete. 2024-04-25T00:56:04.727 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:56:04.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.843+0000 7fe4a7ebe640 1 Processor -- start 2024-04-25T00:56:04.846 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.843+0000 7fe4a7ebe640 1 -- start start 2024-04-25T00:56:04.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a7ebe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4a00affb0 0x7fe4a009da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a7ebe640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a00b1390 con 0x7fe4a009dfa0 2024-04-25T00:56:04.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a7ebe640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a00b1fa0 con 0x7fe4a00b0390 2024-04-25T00:56:04.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a7ebe640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a00b1d70 con 0x7fe4a00affb0 2024-04-25T00:56:04.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a5c5a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe4a00affb0 0x7fe4a009da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a5459640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe4a009dfa0 0x7fe4a0148800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38760/0 (socket says 172.21.15.112:38760) 2024-04-25T00:56:04.847 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a5459640 1 -- 172.21.15.112:0/3624024543 learned_addr learned my addr 172.21.15.112:0/3624024543 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:04.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a5c5a640 1 -- 172.21.15.112:0/3624024543 >> v1:172.21.15.187:6789/0 conn(0x7fe4a00b0390 legacy=0x7fe4a014fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a5c5a640 1 -- 172.21.15.112:0/3624024543 >> v1:172.21.15.112:6789/0 conn(0x7fe4a009dfa0 legacy=0x7fe4a0148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a5c5a640 1 -- 172.21.15.112:0/3624024543 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a00b34b0 con 0x7fe4a00affb0 2024-04-25T00:56:04.849 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a5c5a640 1 --2- 172.21.15.112:0/3624024543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a009da60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0097c0 tx=0x7fe49c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1edcaec3d96c3a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a4c58640 1 -- 172.21.15.112:0/3624024543 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0b0070 con 0x7fe4a00affb0 2024-04-25T00:56:04.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a4c58640 1 -- 172.21.15.112:0/3624024543 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49c0ab600 con 0x7fe4a00affb0 2024-04-25T00:56:04.850 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.847+0000 7fe4a4c58640 1 -- 172.21.15.112:0/3624024543 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c0ab9d0 con 0x7fe4a00affb0 2024-04-25T00:56:04.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/3624024543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 msgr2=0x7fe4a009da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 --2- 172.21.15.112:0/3624024543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a009da60 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0097c0 tx=0x7fe49c01c410 comp rx=0 tx=0).stop 2024-04-25T00:56:04.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/3624024543 shutdown_connections 2024-04-25T00:56:04.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 --2- 172.21.15.112:0/3624024543 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a009da60 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/3624024543 >> 172.21.15.112:0/3624024543 conn(0x7fe4a00b8c80 msgr2=0x7fe4a00b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:04.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/3624024543 shutdown_connections 2024-04-25T00:56:04.851 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/3624024543 wait complete. 2024-04-25T00:56:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 Processor -- start 2024-04-25T00:56:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- start start 2024-04-25T00:56:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a009dfa0 0x7fe4a009d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a0143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a00b0390 0x7fe4a01e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a0153d40 con 0x7fe4a00affb0 2024-04-25T00:56:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a0153bc0 con 0x7fe4a009dfa0 2024-04-25T00:56:04.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a0153ec0 con 0x7fe4a00b0390 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a0143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a0143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60620/0 (socket says 172.21.15.112:60620) 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 -- 172.21.15.112:0/2982767131 learned_addr learned my addr 172.21.15.112:0/2982767131 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5c5a640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a009dfa0 0x7fe4a009d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a645b640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a00b0390 0x7fe4a01e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 -- 172.21.15.112:0/2982767131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a00b0390 msgr2=0x7fe4a01e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a00b0390 0x7fe4a01e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 -- 172.21.15.112:0/2982767131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a009dfa0 msgr2=0x7fe4a009d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a009dfa0 0x7fe4a009d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 -- 172.21.15.112:0/2982767131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a0142f80 con 0x7fe4a00affb0 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a5459640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a0143480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe49800cbc0 tx=0x7fe49800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4927fc640 1 -- 172.21.15.112:0/2982767131 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498017070 con 0x7fe4a00affb0 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4927fc640 1 -- 172.21.15.112:0/2982767131 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49800aca0 con 0x7fe4a00affb0 2024-04-25T00:56:04.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4927fc640 1 -- 172.21.15.112:0/2982767131 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498012600 con 0x7fe4a00affb0 2024-04-25T00:56:04.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a009c960 con 0x7fe4a00affb0 2024-04-25T00:56:04.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a01e20c0 con 0x7fe4a00affb0 2024-04-25T00:56:04.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.851+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a013e7b0 con 0x7fe4a00affb0 2024-04-25T00:56:04.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.855+0000 7fe4927fc640 1 -- 172.21.15.112:0/2982767131 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe49801a070 con 0x7fe4a00affb0 2024-04-25T00:56:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.855+0000 7fe4927fc640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4840678c0 0x7fe484069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.855+0000 7fe4a5c5a640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4840678c0 0x7fe484069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.855+0000 7fe4927fc640 1 -- 172.21.15.112:0/2982767131 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe498085860 con 0x7fe4a00affb0 2024-04-25T00:56:04.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.855+0000 7fe4a5c5a640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4840678c0 0x7fe484069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0ad000 tx=0x7fe49c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:04.859 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:04.859+0000 7fe4927fc640 1 -- 172.21.15.112:0/2982767131 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe498055de0 con 0x7fe4a00affb0 2024-04-25T00:56:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.007+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe4a009cb30 con 0x7fe4a00affb0 2024-04-25T00:56:05.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.007+0000 7fe4927fc640 1 -- 172.21.15.112:0/2982767131 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fe498085070 con 0x7fe4a00affb0 2024-04-25T00:56:05.008 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:56:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4840678c0 msgr2=0x7fe484069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4840678c0 0x7fe484069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe49c0ad000 tx=0x7fe49c008040 comp rx=0 tx=0).stop 2024-04-25T00:56:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 msgr2=0x7fe4a0143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a0143480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe49800cbc0 tx=0x7fe49800a060 comp rx=0 tx=0).stop 2024-04-25T00:56:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 shutdown_connections 2024-04-25T00:56:05.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe4a00b0390 0x7fe4a01e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe4a00affb0 0x7fe4a0143480 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe4840678c0 0x7fe484069d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 --2- 172.21.15.112:0/2982767131 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe4a009dfa0 0x7fe4a009d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 >> 172.21.15.112:0/2982767131 conn(0x7fe4a00b8c80 msgr2=0x7fe4a006bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 shutdown_connections 2024-04-25T00:56:05.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.011+0000 7fe4a7ebe640 1 -- 172.21.15.112:0/2982767131 wait complete. 2024-04-25T00:56:05.027 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:56:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f16329d9640 1 Processor -- start 2024-04-25T00:56:05.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f16329d9640 1 -- start start 2024-04-25T00:56:05.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f16329d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c145200 0x7f162c0b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f16329d9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f162c0aeec0 con 0x7f162c0add70 2024-04-25T00:56:05.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f16329d9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f162c0af780 con 0x7f162c0b23f0 2024-04-25T00:56:05.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f16329d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f162c0b27d0 con 0x7f162c145200 2024-04-25T00:56:05.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162b7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f162c0add70 0x7f162c14db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38772/0 (socket says 172.21.15.112:38772) 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162b7fe640 1 -- 172.21.15.112:0/1592660406 learned_addr learned my addr 172.21.15.112:0/1592660406 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f1630f76640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c145200 0x7f162c0b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1956855157 0 0) 0x7f162c0aeec0 con 0x7f162c0add70 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1614003530 con 0x7f162c0add70 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 17832987 0 0) 0x7f1614003530 con 0x7f162c0add70 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c145200 msgr2=0x7f162c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c145200 0x7f162c0b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 >> v1:172.21.15.187:6789/0 conn(0x7f162c0b23f0 legacy=0x7f162c072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c0afa20 con 0x7f162c0add70 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f16200016a0 con 0x7f162c0add70 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 >> v1:172.21.15.112:6789/0 conn(0x7f162c0add70 legacy=0x7f162c14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16140039f0 0x7f1614003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c145200 0x7f1614004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614004d50 0x7f16140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0b27d0 con 0x7f1614004d50 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f161400b9d0 con 0x7f16140039f0 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f161400bbd0 con 0x7f162c145200 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f1630f76640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614004d50 0x7f16140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162bfff640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16140039f0 0x7f1614003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f162b7fe640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c145200 0x7f1614004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f1630f76640 1 -- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c145200 msgr2=0x7f1614004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f1630f76640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c145200 0x7f1614004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f1630f76640 1 -- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16140039f0 msgr2=0x7f1614003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f1630f76640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16140039f0 0x7f1614003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.147+0000 7f1630f76640 1 -- 172.21.15.112:0/1592660406 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1614003530 con 0x7f1614004d50 2024-04-25T00:56:05.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f1630f76640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614004d50 0x7f16140051d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1624009fc0 tx=0x7f162401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe724870881e9acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240b2070 con 0x7f1614004d50 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16240a5c90 con 0x7f1614004d50 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162affd640 1 -- 172.21.15.112:0/1592660406 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624009c10 con 0x7f1614004d50 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- 172.21.15.112:0/1592660406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614004d50 msgr2=0x7f16140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 --2- 172.21.15.112:0/1592660406 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1614004d50 0x7f16140051d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1624009fc0 tx=0x7f162401cc80 comp rx=0 tx=0).stop 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f1630f76640 1 -- 172.21.15.112:0/1592660406 reap_dead start 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- 172.21.15.112:0/1592660406 shutdown_connections 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- 172.21.15.112:0/1592660406 >> 172.21.15.112:0/1592660406 conn(0x7f162c0b5980 msgr2=0x7f162c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- 172.21.15.112:0/1592660406 shutdown_connections 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- 172.21.15.112:0/1592660406 wait complete. 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 Processor -- start 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- start start 2024-04-25T00:56:05.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f162c140c70 0x7f162c1402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f162c1daeb0 0x7f162c141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c1db5a0 0x7f162c1dba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f162c0b5150 con 0x7f162c1daeb0 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f162c072e70 con 0x7f162c140c70 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f16329d9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f162c0b52d0 con 0x7f162c1db5a0 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f162c1daeb0 0x7f162c141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f162c1daeb0 0x7f162c141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60634/0 (socket says 172.21.15.112:60634) 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162b7fe640 1 -- 172.21.15.112:0/3579531420 learned_addr learned my addr 172.21.15.112:0/3579531420 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162bfff640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f162c140c70 0x7f162c1402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f1630f76640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c1db5a0 0x7f162c1dba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162b7fe640 1 -- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c1db5a0 msgr2=0x7f162c1dba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162b7fe640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c1db5a0 0x7f162c1dba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162b7fe640 1 -- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f162c140c70 msgr2=0x7f162c1402e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162b7fe640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f162c140c70 0x7f162c1402e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f162b7fe640 1 -- 172.21.15.112:0/3579531420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c140820 con 0x7f162c1daeb0 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.151+0000 7f1630f76640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c1db5a0 0x7f162c1dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f162b7fe640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f162c1daeb0 0x7f162c141050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1620010ce0 tx=0x7f162000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f1628ff9640 1 -- 172.21.15.112:0/3579531420 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162001b070 con 0x7f162c1daeb0 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f1628ff9640 1 -- 172.21.15.112:0/3579531420 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1620011040 con 0x7f162c1daeb0 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c142cd0 con 0x7f162c1daeb0 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f1628ff9640 1 -- 172.21.15.112:0/3579531420 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200145a0 con 0x7f162c1daeb0 2024-04-25T00:56:05.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c148980 con 0x7f162c1daeb0 2024-04-25T00:56:05.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f162c04f580 con 0x7f162c1daeb0 2024-04-25T00:56:05.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f1628ff9640 1 -- 172.21.15.112:0/3579531420 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f162001e070 con 0x7f162c1daeb0 2024-04-25T00:56:05.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f1628ff9640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16180678c0 0x7f1618069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f162bfff640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16180678c0 0x7f1618069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f1628ff9640 1 -- 172.21.15.112:0/3579531420 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1620088980 con 0x7f162c1daeb0 2024-04-25T00:56:05.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.155+0000 7f162bfff640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16180678c0 0x7f1618069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f161c000f70 tx=0x7f161c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.159+0000 7f1628ff9640 1 -- 172.21.15.112:0/3579531420 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1620059550 con 0x7f162c1daeb0 2024-04-25T00:56:05.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.331+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f162c0b7270 con 0x7f162c1daeb0 2024-04-25T00:56:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.331+0000 7f1628ff9640 1 -- 172.21.15.112:0/3579531420 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f1620058e50 con 0x7f162c1daeb0 2024-04-25T00:56:05.333 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-04-25T00:56:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16180678c0 msgr2=0x7f1618069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16180678c0 0x7f1618069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f161c000f70 tx=0x7f161c009040 comp rx=0 tx=0).stop 2024-04-25T00:56:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f162c1daeb0 msgr2=0x7f162c141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f162c1daeb0 0x7f162c141050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1620010ce0 tx=0x7f162000e4d0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.335 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 shutdown_connections 2024-04-25T00:56:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f162c1db5a0 0x7f162c1dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f162c1daeb0 0x7f162c141050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16180678c0 0x7f1618069d80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 --2- 172.21.15.112:0/3579531420 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f162c140c70 0x7f162c1402e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 >> 172.21.15.112:0/3579531420 conn(0x7f162c0b5980 msgr2=0x7f162c0726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:05.336 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 shutdown_connections 2024-04-25T00:56:05.337 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.335+0000 7f16329d9640 1 -- 172.21.15.112:0/3579531420 wait complete. 2024-04-25T00:56:05.352 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2024-04-25T00:56:05.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:05.352 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:05.370 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:56:05.370 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:56:05.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:05.370 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:56:05.427 INFO:tasks.cephfs.cephfs_test_case:[] 2024-04-25T00:56:05.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:05.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5aa12f7640 1 Processor -- start 2024-04-25T00:56:05.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5aa12f7640 1 -- start start 2024-04-25T00:56:05.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5aa12f7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a9c0671b0 0x7f5a9c1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5aa12f7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5a9c14b610 0x7f5a9c151ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5aa12f7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a9c067590 con 0x7f5a9c141820 2024-04-25T00:56:05.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5aa12f7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5a9c071330 con 0x7f5a9c14b610 2024-04-25T00:56:05.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5aa12f7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5a9c071610 con 0x7f5a9c0671b0 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a927fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5a9c141820 0x7f5a9c14af00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38774/0 (socket says 172.21.15.112:38774) 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a9b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5a9c14b610 0x7f5a9c151ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a927fc640 1 -- 172.21.15.112:0/648621577 learned_addr learned my addr 172.21.15.112:0/648621577 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a9affd640 1 --2- 172.21.15.112:0/648621577 >> v2:172.21.15.187:3300/0 conn(0x7f5a9c0671b0 0x7f5a9c1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a9b7fe640 1 -- 172.21.15.112:0/648621577 >> v2:172.21.15.187:3300/0 conn(0x7f5a9c0671b0 msgr2=0x7f5a9c1412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a9b7fe640 1 --2- 172.21.15.112:0/648621577 >> v2:172.21.15.187:3300/0 conn(0x7f5a9c0671b0 0x7f5a9c1412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a9b7fe640 1 -- 172.21.15.112:0/648621577 >> v1:172.21.15.112:6789/0 conn(0x7f5a9c141820 legacy=0x7f5a9c14af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a9b7fe640 1 -- 172.21.15.112:0/648621577 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c072010 con 0x7f5a9c14b610 2024-04-25T00:56:05.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a9affd640 1 --2- 172.21.15.112:0/648621577 >> v2:172.21.15.187:3300/0 conn(0x7f5a9c0671b0 0x7f5a9c1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:05.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.595+0000 7f5a9b7fe640 1 --2- 172.21.15.112:0/648621577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c14b610 0x7f5a9c151ed0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0677d0 tx=0x7f5a9401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4cf307849cdd63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9a7fc640 1 -- 172.21.15.112:0/648621577 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a940b2070 con 0x7f5a9c14b610 2024-04-25T00:56:05.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9a7fc640 1 -- 172.21.15.112:0/648621577 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a940a5810 con 0x7f5a9c14b610 2024-04-25T00:56:05.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9a7fc640 1 -- 172.21.15.112:0/648621577 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a940a5be0 con 0x7f5a9c14b610 2024-04-25T00:56:05.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- 172.21.15.112:0/648621577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c14b610 msgr2=0x7f5a9c151ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/648621577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c14b610 0x7f5a9c151ed0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0677d0 tx=0x7f5a9401c8a0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- 172.21.15.112:0/648621577 shutdown_connections 2024-04-25T00:56:05.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/648621577 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c14b610 0x7f5a9c151ed0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/648621577 >> v2:172.21.15.187:3300/0 conn(0x7f5a9c0671b0 0x7f5a9c1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- 172.21.15.112:0/648621577 >> 172.21.15.112:0/648621577 conn(0x7f5a9c06b210 msgr2=0x7f5a9c06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:05.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- 172.21.15.112:0/648621577 shutdown_connections 2024-04-25T00:56:05.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- 172.21.15.112:0/648621577 wait complete. 2024-04-25T00:56:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 Processor -- start 2024-04-25T00:56:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- start start 2024-04-25T00:56:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a9c0671b0 0x7f5a9c141c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c14b610 0x7f5a9c1e4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c1e3970 0x7f5a9c13dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c154fc0 con 0x7f5a9c1e3970 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9c154e40 con 0x7f5a9c14b610 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a9c155140 con 0x7f5a9c0671b0 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c1e3970 0x7f5a9c13dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c1e3970 0x7f5a9c13dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60652/0 (socket says 172.21.15.112:60652) 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 -- 172.21.15.112:0/2492208279 learned_addr learned my addr 172.21.15.112:0/2492208279 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 -- 172.21.15.112:0/2492208279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a9c0671b0 msgr2=0x7f5a9c141c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9affd640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a9c0671b0 0x7f5a9c141c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a927fc640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c14b610 0x7f5a9c1e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a9c0671b0 0x7f5a9c141c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 -- 172.21.15.112:0/2492208279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c14b610 msgr2=0x7f5a9c1e4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c14b610 0x7f5a9c1e4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 -- 172.21.15.112:0/2492208279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9c144330 con 0x7f5a9c1e3970 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a9b7fe640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c1e3970 0x7f5a9c13dc30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a940b0040 tx=0x7f5a94019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a93fff640 1 -- 172.21.15.112:0/2492208279 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a940b2050 con 0x7f5a9c1e3970 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a93fff640 1 -- 172.21.15.112:0/2492208279 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a940be070 con 0x7f5a9c1e3970 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9c1486e0 con 0x7f5a9c1e3970 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5a93fff640 1 -- 172.21.15.112:0/2492208279 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a940ada90 con 0x7f5a9c1e3970 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.599+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9c144050 con 0x7f5a9c1e3970 2024-04-25T00:56:05.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.603+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a9c04f580 con 0x7f5a9c1e3970 2024-04-25T00:56:05.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.603+0000 7f5a93fff640 1 -- 172.21.15.112:0/2492208279 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a940adc30 con 0x7f5a9c1e3970 2024-04-25T00:56:05.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.603+0000 7f5a93fff640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5a7c0678c0 0x7f5a7c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.603+0000 7f5a9affd640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5a7c0678c0 0x7f5a7c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.603+0000 7f5a93fff640 1 -- 172.21.15.112:0/2492208279 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5a94120e10 con 0x7f5a9c1e3970 2024-04-25T00:56:05.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.603+0000 7f5a9affd640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5a7c0678c0 0x7f5a7c069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c006fd0 tx=0x7f5a8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.607+0000 7f5a93fff640 1 -- 172.21.15.112:0/2492208279 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5a94016520 con 0x7f5a9c1e3970 2024-04-25T00:56:05.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.799+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a9c072c30 con 0x7f5a9c1e3970 2024-04-25T00:56:05.802 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5a93fff640 1 -- 172.21.15.112:0/2492208279 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f5a940f19e0 con 0x7f5a9c1e3970 2024-04-25T00:56:05.802 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:05.803 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:55:38.966915+0000","modified":"2024-04-25T00:55:42.002877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:56:05.803 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5a7c0678c0 msgr2=0x7f5a7c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5a7c0678c0 0x7f5a7c069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c006fd0 tx=0x7f5a8c008040 comp rx=0 tx=0).stop 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c1e3970 msgr2=0x7f5a9c13dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c1e3970 0x7f5a9c13dc30 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a940b0040 tx=0x7f5a94019640 comp rx=0 tx=0).stop 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 shutdown_connections 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a9c1e3970 0x7f5a9c13dc30 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5a7c0678c0 0x7f5a7c069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a9c14b610 0x7f5a9c1e4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 --2- 172.21.15.112:0/2492208279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a9c0671b0 0x7f5a9c141c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 >> 172.21.15.112:0/2492208279 conn(0x7f5a9c06b210 msgr2=0x7f5a9c149230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 shutdown_connections 2024-04-25T00:56:05.805 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.803+0000 7f5aa12f7640 1 -- 172.21.15.112:0/2492208279 wait complete. 2024-04-25T00:56:05.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:05.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.943+0000 7f466e135640 1 Processor -- start 2024-04-25T00:56:05.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.943+0000 7f466e135640 1 -- start start 2024-04-25T00:56:05.946 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f46680add70 0x7f4668145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46680b3ce0 0x7f466814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46680aeec0 con 0x7f46680b23f0 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f46680af780 con 0x7f46680b3ce0 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f46680b27d0 con 0x7f46680add70 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4667fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46680b3ce0 0x7f466814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4667fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f46680b3ce0 0x7f466814fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60662/0 (socket says 172.21.15.112:60662) 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4667fff640 1 -- 172.21.15.112:0/3411433290 learned_addr learned my addr 172.21.15.112:0/3411433290 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4666ffd640 1 --2- 172.21.15.112:0/3411433290 >> v2:172.21.15.187:3300/0 conn(0x7f46680add70 0x7f4668145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4667fff640 1 -- 172.21.15.112:0/3411433290 >> v2:172.21.15.187:3300/0 conn(0x7f46680add70 msgr2=0x7f4668145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4667fff640 1 --2- 172.21.15.112:0/3411433290 >> v2:172.21.15.187:3300/0 conn(0x7f46680add70 0x7f4668145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4667fff640 1 -- 172.21.15.112:0/3411433290 >> v1:172.21.15.187:6789/0 conn(0x7f46680b23f0 legacy=0x7f4668072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:05.947 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4667fff640 1 -- 172.21.15.112:0/3411433290 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46680afa20 con 0x7f46680b3ce0 2024-04-25T00:56:05.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f4667fff640 1 --2- 172.21.15.112:0/3411433290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f466814fb40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f466806b8a0 tx=0x7f466001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d217f5c901473c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f46667fc640 1 -- 172.21.15.112:0/3411433290 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600a5780 con 0x7f46680b3ce0 2024-04-25T00:56:05.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f46667fc640 1 -- 172.21.15.112:0/3411433290 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46600a5920 con 0x7f46680b3ce0 2024-04-25T00:56:05.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f46667fc640 1 -- 172.21.15.112:0/3411433290 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600a5cf0 con 0x7f46680b3ce0 2024-04-25T00:56:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 -- 172.21.15.112:0/3411433290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 msgr2=0x7f466814fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 --2- 172.21.15.112:0/3411433290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f466814fb40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f466806b8a0 tx=0x7f466001c9b0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 -- 172.21.15.112:0/3411433290 shutdown_connections 2024-04-25T00:56:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 --2- 172.21.15.112:0/3411433290 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f466814fb40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 --2- 172.21.15.112:0/3411433290 >> v2:172.21.15.187:3300/0 conn(0x7f46680add70 0x7f4668145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 -- 172.21.15.112:0/3411433290 >> 172.21.15.112:0/3411433290 conn(0x7f46680b5980 msgr2=0x7f46680b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 -- 172.21.15.112:0/3411433290 shutdown_connections 2024-04-25T00:56:05.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.947+0000 7f466e135640 1 -- 172.21.15.112:0/3411433290 wait complete. 2024-04-25T00:56:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 Processor -- start 2024-04-25T00:56:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 -- start start 2024-04-25T00:56:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46680add70 0x7f4668139620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46680b23f0 0x7f4668147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f4668132c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46680b52d0 con 0x7f46680b3ce0 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f46680b5150 con 0x7f46680b23f0 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f46681528a0 con 0x7f46680add70 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f4668132c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f4668132c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60676/0 (socket says 172.21.15.112:60676) 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 -- 172.21.15.112:0/3562326865 learned_addr learned my addr 172.21.15.112:0/3562326865 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f46677fe640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46680add70 0x7f4668139620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4666ffd640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46680b23f0 0x7f4668147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 -- 172.21.15.112:0/3562326865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46680add70 msgr2=0x7f4668139620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46680add70 0x7f4668139620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 -- 172.21.15.112:0/3562326865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46680b23f0 msgr2=0x7f4668147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46680b23f0 0x7f4668147990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 -- 172.21.15.112:0/3562326865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4668147200 con 0x7f46680b3ce0 2024-04-25T00:56:05.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f4667fff640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f4668132c70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f466001a5e0 tx=0x7f4660019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f464ffff640 1 -- 172.21.15.112:0/3562326865 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600bc020 con 0x7f46680b3ce0 2024-04-25T00:56:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f464ffff640 1 -- 172.21.15.112:0/3562326865 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46600b8670 con 0x7f46680b3ce0 2024-04-25T00:56:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4668147470 con 0x7f46680b3ce0 2024-04-25T00:56:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f464ffff640 1 -- 172.21.15.112:0/3562326865 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600b8a60 con 0x7f46680b3ce0 2024-04-25T00:56:05.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4668147ed0 con 0x7f46680b3ce0 2024-04-25T00:56:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466804f580 con 0x7f46680b3ce0 2024-04-25T00:56:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.951+0000 7f464ffff640 1 -- 172.21.15.112:0/3562326865 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46600bf020 con 0x7f46680b3ce0 2024-04-25T00:56:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.955+0000 7f464ffff640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46540678c0 0x7f4654069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.955+0000 7f464ffff640 1 -- 172.21.15.112:0/3562326865 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4660120af0 con 0x7f46680b3ce0 2024-04-25T00:56:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.955+0000 7f46677fe640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46540678c0 0x7f4654069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:05.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.955+0000 7f46677fe640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46540678c0 0x7f4654069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f465c0016c0 tx=0x7f465c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:05.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:05.959+0000 7f464ffff640 1 -- 172.21.15.112:0/3562326865 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f46600c6030 con 0x7f46680b3ce0 2024-04-25T00:56:06.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.123+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4668147640 con 0x7f46680b3ce0 2024-04-25T00:56:06.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f464ffff640 1 -- 172.21.15.112:0/3562326865 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f46600165a0 con 0x7f46680b3ce0 2024-04-25T00:56:06.128 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:06.128 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46540678c0 msgr2=0x7f4654069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46540678c0 0x7f4654069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f465c0016c0 tx=0x7f465c009040 comp rx=0 tx=0).stop 2024-04-25T00:56:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 msgr2=0x7f4668132c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.130 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f4668132c70 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f466001a5e0 tx=0x7f4660019750 comp rx=0 tx=0).stop 2024-04-25T00:56:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 shutdown_connections 2024-04-25T00:56:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46680b3ce0 0x7f4668132c70 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46540678c0 0x7f4654069d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46680b23f0 0x7f4668147990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 --2- 172.21.15.112:0/3562326865 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46680add70 0x7f4668139620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 >> 172.21.15.112:0/3562326865 conn(0x7f46680b5980 msgr2=0x7f466806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 shutdown_connections 2024-04-25T00:56:06.131 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.127+0000 7f466e135640 1 -- 172.21.15.112:0/3562326865 wait complete. 2024-04-25T00:56:06.147 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:55:37.465561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:55:37.942022+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:56:06.148 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:56:06.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feee54ab640 1 Processor -- start 2024-04-25T00:56:06.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feee54ab640 1 -- start start 2024-04-25T00:56:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feee54ab640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee009da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feee54ab640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feee009dfa0 0x7feee0148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feee54ab640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7feee00b0390 0x7feee0148d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feee54ab640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7feee00b1d70 con 0x7feee009dfa0 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feee54ab640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7feee00b1600 con 0x7feee00b0390 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feee54ab640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feee00b1390 con 0x7feee00affb0 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feede7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feee009dfa0 0x7feee0148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feede7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7feee009dfa0 0x7feee0148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60692/0 (socket says 172.21.15.112:60692) 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feede7fc640 1 -- 172.21.15.112:0/2868949731 learned_addr learned my addr 172.21.15.112:0/2868949731 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feedeffd640 1 --2- 172.21.15.112:0/2868949731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee009da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.267+0000 7feedf7fe640 1 --2- 172.21.15.112:0/2868949731 >> v2:172.21.15.187:3300/0 conn(0x7feee00b0390 0x7feee0148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feede7fc640 1 -- 172.21.15.112:0/2868949731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 msgr2=0x7feee009da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feede7fc640 1 --2- 172.21.15.112:0/2868949731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee009da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feede7fc640 1 -- 172.21.15.112:0/2868949731 >> v2:172.21.15.187:3300/0 conn(0x7feee00b0390 msgr2=0x7feee0148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feede7fc640 1 --2- 172.21.15.112:0/2868949731 >> v2:172.21.15.187:3300/0 conn(0x7feee00b0390 0x7feee0148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feede7fc640 1 -- 172.21.15.112:0/2868949731 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee00b1fa0 con 0x7feee009dfa0 2024-04-25T00:56:06.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feedeffd640 1 --2- 172.21.15.112:0/2868949731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee009da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feede7fc640 1 --2- 172.21.15.112:0/2868949731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee009dfa0 0x7feee0148800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feee0072ab0 tx=0x7feed401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e83b37fd6ee1c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feeddffb640 1 -- 172.21.15.112:0/2868949731 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40a57c0 con 0x7feee009dfa0 2024-04-25T00:56:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feeddffb640 1 -- 172.21.15.112:0/2868949731 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feed40a5960 con 0x7feee009dfa0 2024-04-25T00:56:06.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feeddffb640 1 -- 172.21.15.112:0/2868949731 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed40a5d30 con 0x7feee009dfa0 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 -- 172.21.15.112:0/2868949731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee009dfa0 msgr2=0x7feee0148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 --2- 172.21.15.112:0/2868949731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee009dfa0 0x7feee0148800 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feee0072ab0 tx=0x7feed401c940 comp rx=0 tx=0).stop 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 -- 172.21.15.112:0/2868949731 shutdown_connections 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 --2- 172.21.15.112:0/2868949731 >> v2:172.21.15.187:3300/0 conn(0x7feee00b0390 0x7feee0148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 --2- 172.21.15.112:0/2868949731 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee009dfa0 0x7feee0148800 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 --2- 172.21.15.112:0/2868949731 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee009da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 -- 172.21.15.112:0/2868949731 >> 172.21.15.112:0/2868949731 conn(0x7feee00b8c80 msgr2=0x7feee00b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 -- 172.21.15.112:0/2868949731 shutdown_connections 2024-04-25T00:56:06.273 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.271+0000 7feee54ab640 1 -- 172.21.15.112:0/2868949731 wait complete. 2024-04-25T00:56:06.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 Processor -- start 2024-04-25T00:56:06.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 -- start start 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feee009dfa0 0x7feee0147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee01e0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee00b0390 0x7feee01e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7feee0145b20 con 0x7feee00b0390 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7feee01459a0 con 0x7feee009dfa0 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7feee0145ca0 con 0x7feee00affb0 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee00b0390 0x7feee01e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee00b0390 0x7feee01e05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60696/0 (socket says 172.21.15.112:60696) 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 -- 172.21.15.112:0/1736812468 learned_addr learned my addr 172.21.15.112:0/1736812468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feede7fc640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee01e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 -- 172.21.15.112:0/1736812468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 msgr2=0x7feee01e0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee01e0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 -- 172.21.15.112:0/1736812468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feee009dfa0 msgr2=0x7feee0147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feee009dfa0 0x7feee0147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 -- 172.21.15.112:0/1736812468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee00aeaa0 con 0x7feee00b0390 2024-04-25T00:56:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedf7fe640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee00b0390 0x7feee01e05b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feed000d5d0 tx=0x7feed000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feec77fe640 1 -- 172.21.15.112:0/1736812468 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00125f0 con 0x7feee00b0390 2024-04-25T00:56:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feec77fe640 1 -- 172.21.15.112:0/1736812468 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feed0019070 con 0x7feee00b0390 2024-04-25T00:56:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee00aed80 con 0x7feee00b0390 2024-04-25T00:56:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feec77fe640 1 -- 172.21.15.112:0/1736812468 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00129f0 con 0x7feee00b0390 2024-04-25T00:56:06.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee00ada10 con 0x7feee00b0390 2024-04-25T00:56:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee00b1040 con 0x7feee00b0390 2024-04-25T00:56:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feec77fe640 1 -- 172.21.15.112:0/1736812468 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feed0012b90 con 0x7feee00b0390 2024-04-25T00:56:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feec77fe640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7feecc0678c0 0x7feecc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feec77fe640 1 -- 172.21.15.112:0/1736812468 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feed0086040 con 0x7feee00b0390 2024-04-25T00:56:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.275+0000 7feedeffd640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7feecc0678c0 0x7feecc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.279+0000 7feedeffd640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7feecc0678c0 0x7feecc069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feed8006e90 tx=0x7feed8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.279+0000 7feec77fe640 1 -- 172.21.15.112:0/1736812468 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feed0056c10 con 0x7feee00b0390 2024-04-25T00:56:06.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.427+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feee004f5f0 con 0x7feee00b0390 2024-04-25T00:56:06.430 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.427+0000 7feec77fe640 1 -- 172.21.15.112:0/1736812468 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7feed0056510 con 0x7feee00b0390 2024-04-25T00:56:06.430 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7feecc0678c0 msgr2=0x7feecc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7feecc0678c0 0x7feecc069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feed8006e90 tx=0x7feed8008040 comp rx=0 tx=0).stop 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee00b0390 msgr2=0x7feee01e05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee00b0390 0x7feee01e05b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7feed000d5d0 tx=0x7feed000da90 comp rx=0 tx=0).stop 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 shutdown_connections 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feee00b0390 0x7feee01e05b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7feee00affb0 0x7feee01e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7feecc0678c0 0x7feecc069d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 --2- 172.21.15.112:0/1736812468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7feee009dfa0 0x7feee0147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 >> 172.21.15.112:0/1736812468 conn(0x7feee00b8c80 msgr2=0x7feee0143990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:06.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 shutdown_connections 2024-04-25T00:56:06.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.431+0000 7feee54ab640 1 -- 172.21.15.112:0/1736812468 wait complete. 2024-04-25T00:56:06.447 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:56:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.555+0000 7f5582c28640 1 Processor -- start 2024-04-25T00:56:06.558 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.555+0000 7f5582c28640 1 -- start start 2024-04-25T00:56:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f5582c28640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f557c0affb0 0x7f557c09da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f5582c28640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c09dfa0 0x7f557c148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f5582c28640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f557c0b1390 con 0x7f557c0b0390 2024-04-25T00:56:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f5582c28640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f557c0b1d70 con 0x7f557c0affb0 2024-04-25T00:56:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f5582c28640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f557c0b1600 con 0x7f557c09dfa0 2024-04-25T00:56:06.559 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f557bfff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c09dfa0 0x7f557c148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f55809c4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f557c0affb0 0x7f557c09da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f55809c4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f557c0affb0 0x7f557c09da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:60906/0 (socket says 172.21.15.112:60906) 2024-04-25T00:56:06.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f55811c5640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f557c0b0390 0x7f557c145190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:55974/0 (socket says 172.21.15.112:55974) 2024-04-25T00:56:06.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f55809c4640 1 -- 172.21.15.112:0/1581389971 learned_addr learned my addr 172.21.15.112:0/1581389971 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:06.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f55809c4640 1 -- 172.21.15.112:0/1581389971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c09dfa0 msgr2=0x7f557c148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:06.560 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f55809c4640 1 --2- 172.21.15.112:0/1581389971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c09dfa0 0x7f557c148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f55809c4640 1 -- 172.21.15.112:0/1581389971 >> v1:172.21.15.187:6789/0 conn(0x7f557c0b0390 legacy=0x7f557c145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.559+0000 7f55809c4640 1 -- 172.21.15.112:0/1581389971 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c0b1fa0 con 0x7f557c0affb0 2024-04-25T00:56:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55809c4640 1 --2- 172.21.15.112:0/1581389971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c0affb0 0x7f557c09da60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5574009e80 tx=0x7f557401c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa18a17145be3954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f557b7fe640 1 -- 172.21.15.112:0/1581389971 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55740a5840 con 0x7f557c0affb0 2024-04-25T00:56:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f557b7fe640 1 -- 172.21.15.112:0/1581389971 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55740a59e0 con 0x7f557c0affb0 2024-04-25T00:56:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f557b7fe640 1 -- 172.21.15.112:0/1581389971 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55740a5db0 con 0x7f557c0affb0 2024-04-25T00:56:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- 172.21.15.112:0/1581389971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c0affb0 msgr2=0x7f557c09da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 --2- 172.21.15.112:0/1581389971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c0affb0 0x7f557c09da60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5574009e80 tx=0x7f557401c980 comp rx=0 tx=0).stop 2024-04-25T00:56:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- 172.21.15.112:0/1581389971 shutdown_connections 2024-04-25T00:56:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 --2- 172.21.15.112:0/1581389971 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c09dfa0 0x7f557c148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 --2- 172.21.15.112:0/1581389971 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c0affb0 0x7f557c09da60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- 172.21.15.112:0/1581389971 >> 172.21.15.112:0/1581389971 conn(0x7f557c0b8c80 msgr2=0x7f557c0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- 172.21.15.112:0/1581389971 shutdown_connections 2024-04-25T00:56:06.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- 172.21.15.112:0/1581389971 wait complete. 2024-04-25T00:56:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 Processor -- start 2024-04-25T00:56:06.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- start start 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c09dfa0 0x7f557c144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c0affb0 0x7f557c140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f557c0b0390 0x7f557c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f557c152a90 con 0x7f557c0b0390 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f557c149150 con 0x7f557c09dfa0 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f557c152c10 con 0x7f557c0affb0 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f557c0b0390 0x7f557c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f557c0b0390 0x7f557c140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60706/0 (socket says 172.21.15.112:60706) 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 -- 172.21.15.112:0/567390950 learned_addr learned my addr 172.21.15.112:0/567390950 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:06.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55809c4640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c09dfa0 0x7f557c144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f557bfff640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c0affb0 0x7f557c140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 -- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c0affb0 msgr2=0x7f557c140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c0affb0 0x7f557c140860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 -- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c09dfa0 msgr2=0x7f557c144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c09dfa0 0x7f557c144560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 -- 172.21.15.112:0/567390950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c1e1cd0 con 0x7f557c0b0390 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55809c4640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c09dfa0 0x7f557c144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55811c5640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f557c0b0390 0x7f557c140da0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c3e0 tx=0x7f556c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55797fa640 1 -- 172.21.15.112:0/567390950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c0074e0 con 0x7f557c0b0390 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55797fa640 1 -- 172.21.15.112:0/567390950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556c007680 con 0x7f557c0b0390 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c1e1fb0 con 0x7f557c0b0390 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f55797fa640 1 -- 172.21.15.112:0/567390950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c0049f0 con 0x7f557c0b0390 2024-04-25T00:56:06.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.563+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c13a570 con 0x7f557c0b0390 2024-04-25T00:56:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.567+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c072640 con 0x7f557c0b0390 2024-04-25T00:56:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.567+0000 7f55797fa640 1 -- 172.21.15.112:0/567390950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f556c01a020 con 0x7f557c0b0390 2024-04-25T00:56:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.567+0000 7f55797fa640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f555c0678c0 0x7f555c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.567+0000 7f55809c4640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f555c0678c0 0x7f555c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.567+0000 7f55797fa640 1 -- 172.21.15.112:0/567390950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f556c085390 con 0x7f557c0b0390 2024-04-25T00:56:06.570 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.567+0000 7f55809c4640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f555c0678c0 0x7f555c069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f55740ae000 tx=0x7f55740a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.572 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.571+0000 7f55797fa640 1 -- 172.21.15.112:0/567390950 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f556c055f60 con 0x7f557c0b0390 2024-04-25T00:56:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f557c0b1040 con 0x7f557c0b0390 2024-04-25T00:56:06.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f55797fa640 1 -- 172.21.15.112:0/567390950 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f556c055860 con 0x7f557c0b0390 2024-04-25T00:56:06.728 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:56:06.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f555c0678c0 msgr2=0x7f555c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f555c0678c0 0x7f555c069d80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f55740ae000 tx=0x7f55740a7070 comp rx=0 tx=0).stop 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f557c0b0390 msgr2=0x7f557c140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f557c0b0390 0x7f557c140da0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f556c00c3e0 tx=0x7f556c00c8a0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 shutdown_connections 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f557c0b0390 0x7f557c140da0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f557c0affb0 0x7f557c140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f555c0678c0 0x7f555c069d80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 --2- 172.21.15.112:0/567390950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f557c09dfa0 0x7f557c144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 >> 172.21.15.112:0/567390950 conn(0x7f557c0b8c80 msgr2=0x7f557c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 shutdown_connections 2024-04-25T00:56:06.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.727+0000 7f5582c28640 1 -- 172.21.15.112:0/567390950 wait complete. 2024-04-25T00:56:06.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:56:06.868 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa11de70640 1 Processor -- start 2024-04-25T00:56:06.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa11de70640 1 -- start start 2024-04-25T00:56:06.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa11de70640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa118148f10 0x7fa118150a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa11de70640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1180b1390 con 0x7fa1180b0390 2024-04-25T00:56:06.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa11de70640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1180b1fa0 con 0x7fa1180affb0 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa11de70640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa1180b1d70 con 0x7fa118148f10 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa116ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa1180b0390 0x7fa118148800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38790/0 (socket says 172.21.15.112:38790) 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa116ffd640 1 -- 172.21.15.112:0/465269958 learned_addr learned my addr 172.21.15.112:0/465269958 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597976732 0 0) 0x7fa1180b1390 con 0x7fa1180b0390 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa100003530 con 0x7fa1180b0390 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.867+0000 7fa117fff640 1 --2- 172.21.15.112:0/465269958 >> v2:172.21.15.187:3300/0 conn(0x7fa118148f10 0x7fa118150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1411919093 0 0) 0x7fa100003530 con 0x7fa1180b0390 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 >> v2:172.21.15.187:3300/0 conn(0x7fa118148f10 msgr2=0x7fa118150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 --2- 172.21.15.112:0/465269958 >> v2:172.21.15.187:3300/0 conn(0x7fa118148f10 0x7fa118150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 >> v1:172.21.15.187:6789/0 conn(0x7fa1180affb0 legacy=0x7fa11809da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1180b34b0 con 0x7fa1180b0390 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7fa10c001680 con 0x7fa1180b0390 2024-04-25T00:56:06.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 >> v1:172.21.15.112:6789/0 conn(0x7fa1180b0390 legacy=0x7fa118148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa100003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa100004810 0x7fa100004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa100005220 0x7fa10000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa10000c140 con 0x7fa1000039c0 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa10000c2f0 con 0x7fa100004810 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa10000c4f0 con 0x7fa100005220 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1177fe640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa100003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa116ffd640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa100004810 0x7fa100004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa117fff640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa100005220 0x7fa10000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1177fe640 1 -- 172.21.15.112:0/465269958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa100005220 msgr2=0x7fa10000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1177fe640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa100005220 0x7fa10000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1177fe640 1 -- 172.21.15.112:0/465269958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa100004810 msgr2=0x7fa100004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1177fe640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa100004810 0x7fa100004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa117fff640 1 -- 172.21.15.112:0/465269958 reap_dead start 2024-04-25T00:56:06.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1177fe640 1 -- 172.21.15.112:0/465269958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1180b34b0 con 0x7fa1000039c0 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1177fe640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa100003de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa108001780 tx=0x7fa10801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc67d3a0175e721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1080013a0 con 0x7fa1000039c0 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1080052f0 con 0x7fa1000039c0 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.871+0000 7fa1167fc640 1 -- 172.21.15.112:0/465269958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1080a4e60 con 0x7fa1000039c0 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- 172.21.15.112:0/465269958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 msgr2=0x7fa100003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa100003de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa108001780 tx=0x7fa10801c3c0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- 172.21.15.112:0/465269958 shutdown_connections 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 --2- 172.21.15.112:0/465269958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa100003de0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- 172.21.15.112:0/465269958 >> 172.21.15.112:0/465269958 conn(0x7fa1180b8c80 msgr2=0x7fa1180b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:06.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- 172.21.15.112:0/465269958 shutdown_connections 2024-04-25T00:56:06.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- 172.21.15.112:0/465269958 wait complete. 2024-04-25T00:56:06.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 Processor -- start 2024-04-25T00:56:06.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- start start 2024-04-25T00:56:06.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa1181481f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1181466e0 0x7fa11809c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11809ce60 0x7fa118134c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa118145310 con 0x7fa1000039c0 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa118145190 con 0x7fa1181466e0 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa118145490 con 0x7fa11809ce60 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa1181481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa1181481f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60724/0 (socket says 172.21.15.112:60724) 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 -- 172.21.15.112:0/3697562428 learned_addr learned my addr 172.21.15.112:0/3697562428 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa116ffd640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1181466e0 0x7fa11809c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 -- 172.21.15.112:0/3697562428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11809ce60 msgr2=0x7fa118134c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11809ce60 0x7fa118134c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 -- 172.21.15.112:0/3697562428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1181466e0 msgr2=0x7fa11809c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:06.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1181466e0 0x7fa11809c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 -- 172.21.15.112:0/3697562428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11809d2e0 con 0x7fa1000039c0 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa116ffd640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1181466e0 0x7fa11809c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa1177fe640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa1181481f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa108005040 tx=0x7fa100005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa0fffff640 1 -- 172.21.15.112:0/3697562428 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1080ba020 con 0x7fa1000039c0 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa0fffff640 1 -- 172.21.15.112:0/3697562428 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1080054a0 con 0x7fa1000039c0 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11809c120 con 0x7fa1000039c0 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa0fffff640 1 -- 172.21.15.112:0/3697562428 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108007a40 con 0x7fa1000039c0 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa118142df0 con 0x7fa1000039c0 2024-04-25T00:56:06.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.875+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1180b1040 con 0x7fa1000039c0 2024-04-25T00:56:06.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.879+0000 7fa0fffff640 1 -- 172.21.15.112:0/3697562428 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa1080bd020 con 0x7fa1000039c0 2024-04-25T00:56:06.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.879+0000 7fa0fffff640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1040678f0 0x7fa104069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:06.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.879+0000 7fa0fffff640 1 -- 172.21.15.112:0/3697562428 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa10811fc20 con 0x7fa1000039c0 2024-04-25T00:56:06.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.879+0000 7fa116ffd640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1040678f0 0x7fa104069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:06.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.879+0000 7fa116ffd640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1040678f0 0x7fa104069db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa10c001ec0 tx=0x7fa10c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:06.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:06.883+0000 7fa0fffff640 1 -- 172.21.15.112:0/3697562428 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa1080c4030 con 0x7fa1000039c0 2024-04-25T00:56:07.045 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.043+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa11804f5f0 con 0x7fa1000039c0 2024-04-25T00:56:07.046 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa0fffff640 1 -- 172.21.15.112:0/3697562428 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa11809d2e0 con 0x7fa1000039c0 2024-04-25T00:56:07.046 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:56:07.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1040678f0 msgr2=0x7fa104069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1040678f0 0x7fa104069db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa10c001ec0 tx=0x7fa10c009040 comp rx=0 tx=0).stop 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 msgr2=0x7fa1181481f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa1181481f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa108005040 tx=0x7fa100005840 comp rx=0 tx=0).stop 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 shutdown_connections 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa11809ce60 0x7fa118134c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa1040678f0 0x7fa104069db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa1181466e0 0x7fa11809c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 --2- 172.21.15.112:0/3697562428 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1000039c0 0x7fa1181481f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 >> 172.21.15.112:0/3697562428 conn(0x7fa1180b8c80 msgr2=0x7fa11813ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 shutdown_connections 2024-04-25T00:56:07.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.047+0000 7fa11de70640 1 -- 172.21.15.112:0/3697562428 wait complete. 2024-04-25T00:56:07.065 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.179+0000 7fa960be3640 1 Processor -- start 2024-04-25T00:56:07.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.179+0000 7fa960be3640 1 -- start start 2024-04-25T00:56:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa95c0b23f0 0x7fa95c0add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa95c145200 0x7fa95c0b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95c0aeec0 con 0x7fa95c072760 2024-04-25T00:56:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa95c0af780 con 0x7fa95c0b23f0 2024-04-25T00:56:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa95c0b27d0 con 0x7fa95c145200 2024-04-25T00:56:07.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa95a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa95c0b23f0 0x7fa95c0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa95ad9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa95c072760 0x7fa95c06c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38796/0 (socket says 172.21.15.112:38796) 2024-04-25T00:56:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa95ad9d640 1 -- 172.21.15.112:0/2369692071 learned_addr learned my addr 172.21.15.112:0/2369692071 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa951d9b640 1 --2- 172.21.15.112:0/2369692071 >> v2:172.21.15.187:3300/0 conn(0x7fa95c145200 0x7fa95c0b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa95a59c640 1 -- 172.21.15.112:0/2369692071 >> v2:172.21.15.187:3300/0 conn(0x7fa95c145200 msgr2=0x7fa95c0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa95a59c640 1 --2- 172.21.15.112:0/2369692071 >> v2:172.21.15.187:3300/0 conn(0x7fa95c145200 0x7fa95c0b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa95a59c640 1 -- 172.21.15.112:0/2369692071 >> v1:172.21.15.112:6789/0 conn(0x7fa95c072760 legacy=0x7fa95c06c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa959d9b640 1 -- 172.21.15.112:0/2369692071 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886209603 0 0) 0x7fa95c0aeec0 con 0x7fa95c072760 2024-04-25T00:56:07.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa95a59c640 1 -- 172.21.15.112:0/2369692071 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c0afa20 con 0x7fa95c0b23f0 2024-04-25T00:56:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa95a59c640 1 --2- 172.21.15.112:0/2369692071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c0add70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0aece0 tx=0x7fa94c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacb30d8b8be9aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa959d9b640 1 -- 172.21.15.112:0/2369692071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0b1070 con 0x7fa95c0b23f0 2024-04-25T00:56:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa959d9b640 1 -- 172.21.15.112:0/2369692071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c0ac5f0 con 0x7fa95c0b23f0 2024-04-25T00:56:07.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa959d9b640 1 -- 172.21.15.112:0/2369692071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0ac9c0 con 0x7fa95c0b23f0 2024-04-25T00:56:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 -- 172.21.15.112:0/2369692071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 msgr2=0x7fa95c0add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 --2- 172.21.15.112:0/2369692071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c0add70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0aece0 tx=0x7fa94c01c480 comp rx=0 tx=0).stop 2024-04-25T00:56:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 -- 172.21.15.112:0/2369692071 shutdown_connections 2024-04-25T00:56:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 --2- 172.21.15.112:0/2369692071 >> v2:172.21.15.187:3300/0 conn(0x7fa95c145200 0x7fa95c0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 --2- 172.21.15.112:0/2369692071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c0add70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 -- 172.21.15.112:0/2369692071 >> 172.21.15.112:0/2369692071 conn(0x7fa95c0b5980 msgr2=0x7fa95c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 -- 172.21.15.112:0/2369692071 shutdown_connections 2024-04-25T00:56:07.187 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.183+0000 7fa960be3640 1 -- 172.21.15.112:0/2369692071 wait complete. 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 Processor -- start 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 -- start start 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa95c072760 0x7fa95c070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa95c145200 0x7fa95c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c072f50 con 0x7fa95c0b23f0 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c0b5360 con 0x7fa95c145200 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa95c152d70 con 0x7fa95c072760 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.189 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c142770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60736/0 (socket says 172.21.15.112:60736) 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 -- 172.21.15.112:0/499281456 learned_addr learned my addr 172.21.15.112:0/499281456 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa95ad9d640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa95c145200 0x7fa95c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 -- 172.21.15.112:0/499281456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa95c072760 msgr2=0x7fa95c070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa95c072760 0x7fa95c070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 -- 172.21.15.112:0/499281456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa95c145200 msgr2=0x7fa95c06f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa95c145200 0x7fa95c06f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 -- 172.21.15.112:0/499281456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c137c50 con 0x7fa95c0b23f0 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa951d9b640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c142770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d620 tx=0x7fa94400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa9537fe640 1 -- 172.21.15.112:0/499281456 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440125f0 con 0x7fa95c0b23f0 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa9537fe640 1 -- 172.21.15.112:0/499281456 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa944019070 con 0x7fa95c0b23f0 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95c071340 con 0x7fa95c0b23f0 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa9537fe640 1 -- 172.21.15.112:0/499281456 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9440129f0 con 0x7fa95c0b23f0 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95c0721f0 con 0x7fa95c0b23f0 2024-04-25T00:56:07.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95c072020 con 0x7fa95c0b23f0 2024-04-25T00:56:07.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.187+0000 7fa9537fe640 1 -- 172.21.15.112:0/499281456 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa944012b90 con 0x7fa95c0b23f0 2024-04-25T00:56:07.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.191+0000 7fa9537fe640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa93c0678c0 0x7fa93c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.191+0000 7fa9537fe640 1 -- 172.21.15.112:0/499281456 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa944086030 con 0x7fa95c0b23f0 2024-04-25T00:56:07.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.191+0000 7fa95a59c640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa93c0678c0 0x7fa93c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.191+0000 7fa95a59c640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa93c0678c0 0x7fa93c069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0ae000 tx=0x7fa94c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.194 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.195+0000 7fa9537fe640 1 -- 172.21.15.112:0/499281456 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa95c137c50 con 0x7fa95c0b23f0 2024-04-25T00:56:07.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.371+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa95c150810 con 0x7fa95c0b23f0 2024-04-25T00:56:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa9537fe640 1 -- 172.21.15.112:0/499281456 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7fa95c150810 con 0x7fa95c0b23f0 2024-04-25T00:56:07.375 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:07.375 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:55:38.966915+0000","modified":"2024-04-25T00:55:42.002877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:56:07.375 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T00:56:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa93c0678c0 msgr2=0x7fa93c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa93c0678c0 0x7fa93c069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0ae000 tx=0x7fa94c008040 comp rx=0 tx=0).stop 2024-04-25T00:56:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 msgr2=0x7fa95c142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c142770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa94400d620 tx=0x7fa94400dae0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 shutdown_connections 2024-04-25T00:56:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa93c0678c0 0x7fa93c069d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa95c145200 0x7fa95c06f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa95c0b23f0 0x7fa95c142770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 --2- 172.21.15.112:0/499281456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa95c072760 0x7fa95c070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 >> 172.21.15.112:0/499281456 conn(0x7fa95c0b5980 msgr2=0x7fa95c0b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 shutdown_connections 2024-04-25T00:56:07.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.375+0000 7fa960be3640 1 -- 172.21.15.112:0/499281456 wait complete. 2024-04-25T00:56:07.392 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:07.510 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f364e206640 1 Processor -- start 2024-04-25T00:56:07.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f364e206640 1 -- start start 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f364e206640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3648072400 0x7f36480b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f364e206640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 0x7f36480b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f364e206640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36480727e0 con 0x7f3648148820 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f364e206640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f364813fa70 con 0x7f3648072400 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f364e206640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f364806ed30 con 0x7f36480b7030 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f36477fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3648072400 0x7f36480b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f3647fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3648148820 0x7f36480b54d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38806/0 (socket says 172.21.15.112:38806) 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f3647fff640 1 -- 172.21.15.112:0/1066945784 learned_addr learned my addr 172.21.15.112:0/1066945784 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f363ffff640 1 --2- 172.21.15.112:0/1066945784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 0x7f36480b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f36477fe640 1 -- 172.21.15.112:0/1066945784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 msgr2=0x7f36480b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:07.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f36477fe640 1 --2- 172.21.15.112:0/1066945784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 0x7f36480b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f36477fe640 1 -- 172.21.15.112:0/1066945784 >> v1:172.21.15.112:6789/0 conn(0x7f3648148820 legacy=0x7f36480b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f36477fe640 1 -- 172.21.15.112:0/1066945784 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36480b29a0 con 0x7f3648072400 2024-04-25T00:56:07.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.511+0000 7f3646ffd640 1 -- 172.21.15.112:0/1066945784 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897028993 0 0) 0x7f36480727e0 con 0x7f3648148820 2024-04-25T00:56:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 --2- 172.21.15.112:0/1066945784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f36480b4330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f36300097c0 tx=0x7f363001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a76f8b387a81b2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f3646ffd640 1 -- 172.21.15.112:0/1066945784 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300b0070 con 0x7f3648072400 2024-04-25T00:56:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f3646ffd640 1 -- 172.21.15.112:0/1066945784 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36300ab600 con 0x7f3648072400 2024-04-25T00:56:07.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f3646ffd640 1 -- 172.21.15.112:0/1066945784 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300ab9d0 con 0x7f3648072400 2024-04-25T00:56:07.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- 172.21.15.112:0/1066945784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 msgr2=0x7f36480b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 --2- 172.21.15.112:0/1066945784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f36480b4330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f36300097c0 tx=0x7f363001c410 comp rx=0 tx=0).stop 2024-04-25T00:56:07.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- 172.21.15.112:0/1066945784 shutdown_connections 2024-04-25T00:56:07.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 --2- 172.21.15.112:0/1066945784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 0x7f36480b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 --2- 172.21.15.112:0/1066945784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f36480b4330 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- 172.21.15.112:0/1066945784 >> 172.21.15.112:0/1066945784 conn(0x7f36480b77a0 msgr2=0x7f36480b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:07.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- 172.21.15.112:0/1066945784 shutdown_connections 2024-04-25T00:56:07.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- 172.21.15.112:0/1066945784 wait complete. 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 Processor -- start 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- start start 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f364813b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 0x7f3648140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3648148820 0x7f3648141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f364806e7f0 con 0x7f3648072400 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f364806e670 con 0x7f3648148820 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3648152b70 con 0x7f36480b7030 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f364813b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f364813b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60746/0 (socket says 172.21.15.112:60746) 2024-04-25T00:56:07.518 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 -- 172.21.15.112:0/1325290253 learned_addr learned my addr 172.21.15.112:0/1325290253 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f363ffff640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 0x7f3648140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 -- 172.21.15.112:0/1325290253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 msgr2=0x7f3648140100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 0x7f3648140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 -- 172.21.15.112:0/1325290253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3648148820 msgr2=0x7f3648141790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f3647fff640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3648148820 0x7f3648141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3648148820 0x7f3648141790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 -- 172.21.15.112:0/1325290253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3648141e70 con 0x7f3648072400 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f36477fe640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f364813b610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36300097c0 tx=0x7f363001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f3644ff9640 1 -- 172.21.15.112:0/1325290253 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300b0040 con 0x7f3648072400 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f3644ff9640 1 -- 172.21.15.112:0/1325290253 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3630004030 con 0x7f3648072400 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f3644ff9640 1 -- 172.21.15.112:0/1325290253 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36300043e0 con 0x7f3648072400 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3648142150 con 0x7f3648072400 2024-04-25T00:56:07.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.515+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364813a7f0 con 0x7f3648072400 2024-04-25T00:56:07.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.519+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f364813fcb0 con 0x7f3648072400 2024-04-25T00:56:07.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.519+0000 7f3644ff9640 1 -- 172.21.15.112:0/1325290253 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3630003620 con 0x7f3648072400 2024-04-25T00:56:07.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.519+0000 7f3644ff9640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f362c0678c0 0x7f362c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.519+0000 7f3644ff9640 1 -- 172.21.15.112:0/1325290253 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f36301207c0 con 0x7f3648072400 2024-04-25T00:56:07.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.519+0000 7f363ffff640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f362c0678c0 0x7f362c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.519+0000 7f363ffff640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f362c0678c0 0x7f362c069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f364813eea0 tx=0x7f3638008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.523+0000 7f3644ff9640 1 -- 172.21.15.112:0/1325290253 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f364813fcb0 con 0x7f3648072400 2024-04-25T00:56:07.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.683+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3648140b00 con 0x7f3648072400 2024-04-25T00:56:07.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f3644ff9640 1 -- 172.21.15.112:0/1325290253 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7f3630016090 con 0x7f3648072400 2024-04-25T00:56:07.687 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:07.687 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:07.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f362c0678c0 msgr2=0x7f362c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f362c0678c0 0x7f362c069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f364813eea0 tx=0x7f3638008040 comp rx=0 tx=0).stop 2024-04-25T00:56:07.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 msgr2=0x7f364813b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f364813b610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f36300097c0 tx=0x7f363001ac50 comp rx=0 tx=0).stop 2024-04-25T00:56:07.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 shutdown_connections 2024-04-25T00:56:07.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f362c0678c0 0x7f362c069d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3648148820 0x7f3648141790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f36480b7030 0x7f3648140100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 --2- 172.21.15.112:0/1325290253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3648072400 0x7f364813b610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 >> 172.21.15.112:0/1325290253 conn(0x7f36480b77a0 msgr2=0x7f3648071800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:07.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 shutdown_connections 2024-04-25T00:56:07.690 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.687+0000 7f364e206640 1 -- 172.21.15.112:0/1325290253 wait complete. 2024-04-25T00:56:07.705 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:56:07.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:07.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25ea833640 1 Processor -- start 2024-04-25T00:56:07.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25ea833640 1 -- start start 2024-04-25T00:56:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25ea833640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f25e406c630 0x7f25e40aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25ea833640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e41485e0 con 0x7f25e406ca10 2024-04-25T00:56:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25ea833640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e40af760 con 0x7f25e40add70 2024-04-25T00:56:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25ea833640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f25e4148860 con 0x7f25e406c630 2024-04-25T00:56:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e37fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f25e406ca10 0x7f25e41477c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38808/0 (socket says 172.21.15.112:38808) 2024-04-25T00:56:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e37fe640 1 -- 172.21.15.112:0/477037097 learned_addr learned my addr 172.21.15.112:0/477037097 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:07.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e3fff640 1 --2- 172.21.15.112:0/477037097 >> v2:172.21.15.187:3300/0 conn(0x7f25e406c630 0x7f25e40aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3521382474 0 0) 0x7f25e41485e0 con 0x7f25e406ca10 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f25cc003530 con 0x7f25e406ca10 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 62419718 0 0) 0x7f25cc003530 con 0x7f25e406ca10 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 >> v2:172.21.15.187:3300/0 conn(0x7f25e406c630 msgr2=0x7f25e40aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 --2- 172.21.15.112:0/477037097 >> v2:172.21.15.187:3300/0 conn(0x7f25e406c630 0x7f25e40aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 >> v1:172.21.15.187:6789/0 conn(0x7f25e40add70 legacy=0x7f25e4147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e40afa00 con 0x7f25e406ca10 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f25d8001680 con 0x7f25e406ca10 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 >> v1:172.21.15.112:6789/0 conn(0x7f25e406ca10 legacy=0x7f25e41477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25cc0039c0 0x7f25cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25cc004810 0x7f25cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0053d0 0x7f25cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc00c230 con 0x7f25cc0039c0 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25cc00c3e0 con 0x7f25cc004810 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25cc00c5e0 con 0x7f25cc0053d0 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e3fff640 1 --2- 172.21.15.112:0/477037097 >> v2:172.21.15.187:3300/0 conn(0x7f25e406c630 0x7f25e40aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e3fff640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25cc0039c0 0x7f25cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e37fe640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25cc004810 0x7f25cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e8dd0640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0053d0 0x7f25cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e3fff640 1 -- 172.21.15.112:0/477037097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0053d0 msgr2=0x7f25cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e3fff640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0053d0 0x7f25cc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e3fff640 1 -- 172.21.15.112:0/477037097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25cc004810 msgr2=0x7f25cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e3fff640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25cc004810 0x7f25cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e3fff640 1 -- 172.21.15.112:0/477037097 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25cc003530 con 0x7f25cc0039c0 2024-04-25T00:56:07.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.815+0000 7f25e8dd0640 1 -- 172.21.15.112:0/477037097 reap_dead start 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e3fff640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25cc0039c0 0x7f25cc003de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25d40098b0 tx=0x7f25d401c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7babdf3bb2ec0272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d400f040 con 0x7f25cc0039c0 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d40a8050 con 0x7f25cc0039c0 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e2ffd640 1 -- 172.21.15.112:0/477037097 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d4004440 con 0x7f25cc0039c0 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- 172.21.15.112:0/477037097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25cc0039c0 msgr2=0x7f25cc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25cc0039c0 0x7f25cc003de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25d40098b0 tx=0x7f25d401c5b0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- 172.21.15.112:0/477037097 shutdown_connections 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 --2- 172.21.15.112:0/477037097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25cc0039c0 0x7f25cc003de0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- 172.21.15.112:0/477037097 >> 172.21.15.112:0/477037097 conn(0x7f25e40b5980 msgr2=0x7f25e40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- 172.21.15.112:0/477037097 shutdown_connections 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- 172.21.15.112:0/477037097 wait complete. 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 Processor -- start 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- start start 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0039c0 0x7f25e4143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e41446e0 0x7f25e4136530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e4136a70 0x7f25e4136ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25e4148360 con 0x7f25e4136a70 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25e41481e0 con 0x7f25e41446e0 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25e40b3480 con 0x7f25cc0039c0 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e4136a70 0x7f25e4136ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e4136a70 0x7f25e4136ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60764/0 (socket says 172.21.15.112:60764) 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 -- 172.21.15.112:0/1860703696 learned_addr learned my addr 172.21.15.112:0/1860703696 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:07.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e3fff640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0039c0 0x7f25e4143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e37fe640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e41446e0 0x7f25e4136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 -- 172.21.15.112:0/1860703696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0039c0 msgr2=0x7f25e4143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0039c0 0x7f25e4143f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 -- 172.21.15.112:0/1860703696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e41446e0 msgr2=0x7f25e4136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e41446e0 0x7f25e4136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 -- 172.21.15.112:0/1860703696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e4138570 con 0x7f25e4136a70 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e8dd0640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e4136a70 0x7f25e4136ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25e40afe50 tx=0x7f25dc00d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e0ff9640 1 -- 172.21.15.112:0/1860703696 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc018070 con 0x7f25e4136a70 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e0ff9640 1 -- 172.21.15.112:0/1860703696 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc003d90 con 0x7f25e4136a70 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e4072310 con 0x7f25e4136a70 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25e0ff9640 1 -- 172.21.15.112:0/1860703696 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc005290 con 0x7f25e4136a70 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.819+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e4138290 con 0x7f25e4136a70 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.823+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e404f580 con 0x7f25e4136a70 2024-04-25T00:56:07.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.823+0000 7f25e0ff9640 1 -- 172.21.15.112:0/1860703696 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f25dc01b070 con 0x7f25e4136a70 2024-04-25T00:56:07.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.823+0000 7f25e0ff9640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25d00678c0 0x7f25d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:07.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.823+0000 7f25e0ff9640 1 -- 172.21.15.112:0/1860703696 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f25dc0851f0 con 0x7f25e4136a70 2024-04-25T00:56:07.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.823+0000 7f25e3fff640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25d00678c0 0x7f25d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:07.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.823+0000 7f25e3fff640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25d00678c0 0x7f25d0069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25d40ad000 tx=0x7f25d40046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:07.828 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:07.827+0000 7f25e0ff9640 1 -- 172.21.15.112:0/1860703696 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f25dc055dc0 con 0x7f25e4136a70 2024-04-25T00:56:08.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.007+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25e4148a20 con 0x7f25e4136a70 2024-04-25T00:56:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.007+0000 7f25e0ff9640 1 -- 172.21.15.112:0/1860703696 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f25dc0556c0 con 0x7f25e4136a70 2024-04-25T00:56:08.008 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:08.008 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:55:38.966915+0000","modified":"2024-04-25T00:55:42.002877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:56:08.008 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T00:56:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25d00678c0 msgr2=0x7f25d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25d00678c0 0x7f25d0069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25d40ad000 tx=0x7f25d40046d0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e4136a70 msgr2=0x7f25e4136ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e4136a70 0x7f25e4136ef0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f25e40afe50 tx=0x7f25dc00d950 comp rx=0 tx=0).stop 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 shutdown_connections 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e4136a70 0x7f25e4136ef0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25d00678c0 0x7f25d0069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f25e41446e0 0x7f25e4136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 --2- 172.21.15.112:0/1860703696 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f25cc0039c0 0x7f25e4143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 >> 172.21.15.112:0/1860703696 conn(0x7f25e40b5980 msgr2=0x7f25e413ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 shutdown_connections 2024-04-25T00:56:08.011 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.011+0000 7f25ea833640 1 -- 172.21.15.112:0/1860703696 wait complete. 2024-04-25T00:56:08.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.135+0000 7fc7b07f7640 1 Processor -- start 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7b07f7640 1 -- start start 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7b07f7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 0x7fc7a80aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7b07f7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc7a80add70 0x7fc7a80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7b07f7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7a80af760 con 0x7fc7a806c630 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7b07f7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc7a8148860 con 0x7fc7a80add70 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7b07f7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a80b52f0 con 0x7fc7a806ca10 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7ae593640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc7a806c630 0x7fc7a81477c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38820/0 (socket says 172.21.15.112:38820) 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7ae593640 1 -- 172.21.15.112:0/2320557583 learned_addr learned my addr 172.21.15.112:0/2320557583 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:08.140 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7aed94640 1 --2- 172.21.15.112:0/2320557583 >> v2:172.21.15.112:3300/0 conn(0x7fc7a80add70 0x7fc7a80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7ad591640 1 -- 172.21.15.112:0/2320557583 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1392843758 0 0) 0x7fc7a80af760 con 0x7fc7a806c630 2024-04-25T00:56:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7add92640 1 --2- 172.21.15.112:0/2320557583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 0x7fc7a80aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7ad591640 1 -- 172.21.15.112:0/2320557583 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc794003530 con 0x7fc7a806c630 2024-04-25T00:56:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7aed94640 1 -- 172.21.15.112:0/2320557583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 msgr2=0x7fc7a80aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7aed94640 1 --2- 172.21.15.112:0/2320557583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 0x7fc7a80aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7aed94640 1 -- 172.21.15.112:0/2320557583 >> v1:172.21.15.112:6789/0 conn(0x7fc7a806c630 legacy=0x7fc7a81477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.141 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7aed94640 1 -- 172.21.15.112:0/2320557583 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a80afa00 con 0x7fc7a80add70 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7aed94640 1 --2- 172.21.15.112:0/2320557583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a80add70 0x7fc7a80b4db0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009d90 tx=0x7fc7a001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3956639dc9efa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7ad591640 1 -- 172.21.15.112:0/2320557583 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00ac070 con 0x7fc7a80add70 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7ad591640 1 -- 172.21.15.112:0/2320557583 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a00a6640 con 0x7fc7a80add70 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.139+0000 7fc7ad591640 1 -- 172.21.15.112:0/2320557583 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a00a6a10 con 0x7fc7a80add70 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2320557583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a80add70 msgr2=0x7fc7a80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2320557583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a80add70 0x7fc7a80b4db0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc7a0009d90 tx=0x7fc7a001c850 comp rx=0 tx=0).stop 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2320557583 shutdown_connections 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2320557583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a80add70 0x7fc7a80b4db0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2320557583 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 0x7fc7a80aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2320557583 >> 172.21.15.112:0/2320557583 conn(0x7fc7a80b5980 msgr2=0x7fc7a80b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:08.143 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2320557583 shutdown_connections 2024-04-25T00:56:08.144 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2320557583 wait complete. 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 Processor -- start 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- start start 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a806c630 0x7fc7a8141230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 0x7fc7a809b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7a80add70 0x7fc7a809bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a81519f0 con 0x7fc7a806c630 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a8151870 con 0x7fc7a80add70 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a8151b70 con 0x7fc7a806ca10 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a806c630 0x7fc7a8141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a806c630 0x7fc7a8141230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60794/0 (socket says 172.21.15.112:60794) 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 -- 172.21.15.112:0/2474908052 learned_addr learned my addr 172.21.15.112:0/2474908052 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:08.148 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 -- 172.21.15.112:0/2474908052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 msgr2=0x7fc7a809b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 0x7fc7a809b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 -- 172.21.15.112:0/2474908052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7a80add70 msgr2=0x7fc7a809bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7a80add70 0x7fc7a809bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 -- 172.21.15.112:0/2474908052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a809ce30 con 0x7fc7a806c630 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7ae593640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a806c630 0x7fc7a8141230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc7980010f0 tx=0x7fc79800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc79effd640 1 -- 172.21.15.112:0/2474908052 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798017070 con 0x7fc7a806c630 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc79effd640 1 -- 172.21.15.112:0/2474908052 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc798005d10 con 0x7fc7a806c630 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc79effd640 1 -- 172.21.15.112:0/2474908052 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc798007310 con 0x7fc7a806c630 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a81431e0 con 0x7fc7a806c630 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a809ae40 con 0x7fc7a806c630 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a804f580 con 0x7fc7a806c630 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.143+0000 7fc79effd640 1 -- 172.21.15.112:0/2474908052 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc79801a070 con 0x7fc7a806c630 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.147+0000 7fc79effd640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc78c0678f0 0x7fc78c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.147+0000 7fc7add92640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc78c0678f0 0x7fc78c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.147+0000 7fc79effd640 1 -- 172.21.15.112:0/2474908052 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc798085070 con 0x7fc7a806c630 2024-04-25T00:56:08.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.147+0000 7fc7add92640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc78c0678f0 0x7fc78c069db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8144520 tx=0x7fc7a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.151+0000 7fc79effd640 1 -- 172.21.15.112:0/2474908052 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc798055c40 con 0x7fc7a806c630 2024-04-25T00:56:08.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.327+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7a806d090 con 0x7fc7a806c630 2024-04-25T00:56:08.329 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.327+0000 7fc79effd640 1 -- 172.21.15.112:0/2474908052 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7fc798055540 con 0x7fc7a806c630 2024-04-25T00:56:08.330 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:08.330 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/571871244","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":571871244},{"type":"v1","addr":"172.21.15.112:6835","nonce":571871244}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:55:38.966915+0000","modified":"2024-04-25T00:55:42.002877+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.187:6833/2006003648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":2006003648},{"type":"v1","addr":"172.21.15.187:6833","nonce":2006003648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4270,"qdb_cluster":[4270]},"id":1}]} 2024-04-25T00:56:08.330 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 8 2024-04-25T00:56:08.332 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc78c0678f0 msgr2=0x7fc78c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc78c0678f0 0x7fc78c069db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc7a8144520 tx=0x7fc7a4008040 comp rx=0 tx=0).stop 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a806c630 msgr2=0x7fc7a8141230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a806c630 0x7fc7a8141230 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc7980010f0 tx=0x7fc79800c8f0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 shutdown_connections 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc78c0678f0 0x7fc78c069db0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc7a80add70 0x7fc7a809bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc7a806ca10 0x7fc7a809b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 --2- 172.21.15.112:0/2474908052 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc7a806c630 0x7fc7a8141230 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 >> 172.21.15.112:0/2474908052 conn(0x7fc7a80b5980 msgr2=0x7fc7a80b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 shutdown_connections 2024-04-25T00:56:08.333 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.331+0000 7fc7b07f7640 1 -- 172.21.15.112:0/2474908052 wait complete. 2024-04-25T00:56:08.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.459+0000 7fd5c3367640 1 Processor -- start 2024-04-25T00:56:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.459+0000 7fd5c3367640 1 -- start start 2024-04-25T00:56:08.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c3367640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5bc0b5790 0x7fd5bc150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c3367640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5bc067d70 con 0x7fd5bc0b3a80 2024-04-25T00:56:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c3367640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5bc0b4350 con 0x7fd5bc072530 2024-04-25T00:56:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c3367640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd5bc072910 con 0x7fd5bc0b5790 2024-04-25T00:56:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c1904640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5bc0b5790 0x7fd5bc150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c0902640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd5bc0b3a80 0x7fd5bc067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38828/0 (socket says 172.21.15.112:38828) 2024-04-25T00:56:08.465 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c0902640 1 -- 172.21.15.112:0/4255920826 learned_addr learned my addr 172.21.15.112:0/4255920826 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c1904640 1 -- 172.21.15.112:0/4255920826 >> v1:172.21.15.187:6789/0 conn(0x7fd5bc072530 legacy=0x7fd5bc066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c1904640 1 -- 172.21.15.112:0/4255920826 >> v1:172.21.15.112:6789/0 conn(0x7fd5bc0b3a80 legacy=0x7fd5bc067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c1904640 1 -- 172.21.15.112:0/4255920826 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc0b4c40 con 0x7fd5bc0b5790 2024-04-25T00:56:08.466 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5b3fff640 1 -- 172.21.15.112:0/4255920826 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 64905840 0 0) 0x7fd5bc067d70 con 0x7fd5bc0b3a80 2024-04-25T00:56:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5c1904640 1 --2- 172.21.15.112:0/4255920826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc150ca0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009d90 tx=0x7fd5b401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314b5b354f7573d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5b3fff640 1 -- 172.21.15.112:0/4255920826 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b40a8630 con 0x7fd5bc0b5790 2024-04-25T00:56:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5b3fff640 1 -- 172.21.15.112:0/4255920826 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b40a87d0 con 0x7fd5bc0b5790 2024-04-25T00:56:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.463+0000 7fd5b3fff640 1 -- 172.21.15.112:0/4255920826 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b40a8ba0 con 0x7fd5bc0b5790 2024-04-25T00:56:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- 172.21.15.112:0/4255920826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 msgr2=0x7fd5bc150ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 --2- 172.21.15.112:0/4255920826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc150ca0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4009d90 tx=0x7fd5b401c810 comp rx=0 tx=0).stop 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- 172.21.15.112:0/4255920826 shutdown_connections 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 --2- 172.21.15.112:0/4255920826 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc150ca0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- 172.21.15.112:0/4255920826 >> 172.21.15.112:0/4255920826 conn(0x7fd5bc06b210 msgr2=0x7fd5bc06b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- 172.21.15.112:0/4255920826 shutdown_connections 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- 172.21.15.112:0/4255920826 wait complete. 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 Processor -- start 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- start start 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5bc072530 0x7fd5bc0ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5bc0b3a80 0x7fd5bc142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.472 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc0a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b7250 con 0x7fd5bc0b5790 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b3ed0 con 0x7fd5bc0b3a80 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b73d0 con 0x7fd5bc072530 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc0a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc0a9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60826/0 (socket says 172.21.15.112:60826) 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 -- 172.21.15.112:0/4126259624 learned_addr learned my addr 172.21.15.112:0/4126259624 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 -- 172.21.15.112:0/4126259624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5bc072530 msgr2=0x7fd5bc0ad2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1103640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5bc072530 0x7fd5bc0ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5bc072530 0x7fd5bc0ad2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 -- 172.21.15.112:0/4126259624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5bc0b3a80 msgr2=0x7fd5bc142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5bc0b3a80 0x7fd5bc142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 -- 172.21.15.112:0/4126259624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc0aa370 con 0x7fd5bc0b5790 2024-04-25T00:56:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1103640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5bc072530 0x7fd5bc0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c1904640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc0a9e30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4019420 tx=0x7fd5b40195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5b1ffb640 1 -- 172.21.15.112:0/4126259624 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b40b0810 con 0x7fd5bc0b5790 2024-04-25T00:56:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5b1ffb640 1 -- 172.21.15.112:0/4126259624 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b40bf070 con 0x7fd5bc0b5790 2024-04-25T00:56:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc0b0730 con 0x7fd5bc0b5790 2024-04-25T00:56:08.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5b1ffb640 1 -- 172.21.15.112:0/4126259624 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b40b0bf0 con 0x7fd5bc0b5790 2024-04-25T00:56:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc0a9540 con 0x7fd5bc0b5790 2024-04-25T00:56:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.467+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc04f580 con 0x7fd5bc0b5790 2024-04-25T00:56:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.471+0000 7fd5b1ffb640 1 -- 172.21.15.112:0/4126259624 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd5b40a8620 con 0x7fd5bc0b5790 2024-04-25T00:56:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.471+0000 7fd5b1ffb640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5a00678f0 0x7fd5a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.471+0000 7fd5b1ffb640 1 -- 172.21.15.112:0/4126259624 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd5b41209a0 con 0x7fd5bc0b5790 2024-04-25T00:56:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.471+0000 7fd5c1103640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5a00678f0 0x7fd5a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.471+0000 7fd5c1103640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5a00678f0 0x7fd5a0069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00a850 tx=0x7fd5ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.475+0000 7fd5b1ffb640 1 -- 172.21.15.112:0/4126259624 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd5b40c8020 con 0x7fd5bc0b5790 2024-04-25T00:56:08.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.635+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5bc0b0520 con 0x7fd5bc0b5790 2024-04-25T00:56:08.639 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5b1ffb640 1 -- 172.21.15.112:0/4126259624 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16653 (secure 0 0 0) 0x7fd5bc0aa370 con 0x7fd5bc0b5790 2024-04-25T00:56:08.640 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:08.640 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":20,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:55:40.622669+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:55:37.465561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:55:37.942022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5a00678f0 msgr2=0x7fd5a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5a00678f0 0x7fd5a0069db0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00a850 tx=0x7fd5ac008040 comp rx=0 tx=0).stop 2024-04-25T00:56:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 msgr2=0x7fd5bc0a9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc0a9e30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4019420 tx=0x7fd5b40195b0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 shutdown_connections 2024-04-25T00:56:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5bc0b5790 0x7fd5bc0a9e30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5a00678f0 0x7fd5a0069db0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5bc0b3a80 0x7fd5bc142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 --2- 172.21.15.112:0/4126259624 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5bc072530 0x7fd5bc0ad2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.645 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 >> 172.21.15.112:0/4126259624 conn(0x7fd5bc06b210 msgr2=0x7fd5bc06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:08.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 shutdown_connections 2024-04-25T00:56:08.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.639+0000 7fd5c3367640 1 -- 172.21.15.112:0/4126259624 wait complete. 2024-04-25T00:56:08.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:56:08.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b3259c640 1 Processor -- start 2024-04-25T00:56:08.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b3259c640 1 -- start start 2024-04-25T00:56:08.780 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b3259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c06c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b3259c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5b2c0b8e80 0x7f5b2c0b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b3259c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b2c0b9470 con 0x7f5b2c0bae60 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b3259c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5b2c0bb240 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b3259c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c0b9710 con 0x7f5b2c141a70 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b2bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5b2c0bae60 0x7f5b2c141360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38838/0 (socket says 172.21.15.112:38838) 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b2bfff640 1 -- 172.21.15.112:0/2141327477 learned_addr learned my addr 172.21.15.112:0/2141327477 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b30b39640 1 --2- 172.21.15.112:0/2141327477 >> v2:172.21.15.112:3300/0 conn(0x7f5b2c0b8e80 0x7f5b2c0b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b2b7fe640 1 --2- 172.21.15.112:0/2141327477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c06c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b2affd640 1 -- 172.21.15.112:0/2141327477 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2564309617 0 0) 0x7f5b2c0b9470 con 0x7f5b2c0bae60 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b2affd640 1 -- 172.21.15.112:0/2141327477 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b14003530 con 0x7f5b2c0bae60 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b30b39640 1 -- 172.21.15.112:0/2141327477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 msgr2=0x7f5b2c06c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:08.781 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b30b39640 1 --2- 172.21.15.112:0/2141327477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c06c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b30b39640 1 -- 172.21.15.112:0/2141327477 >> v1:172.21.15.112:6789/0 conn(0x7f5b2c0bae60 legacy=0x7f5b2c141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.779+0000 7f5b30b39640 1 -- 172.21.15.112:0/2141327477 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c0b99f0 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b30b39640 1 --2- 172.21.15.112:0/2141327477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c0b9fc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009d90 tx=0x7f5b1c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4051361e292c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2affd640 1 -- 172.21.15.112:0/2141327477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c0b2070 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2affd640 1 -- 172.21.15.112:0/2141327477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b1c0a5940 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2affd640 1 -- 172.21.15.112:0/2141327477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c06d200 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- 172.21.15.112:0/2141327477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 msgr2=0x7f5b2c0b9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 --2- 172.21.15.112:0/2141327477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c0b9fc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009d90 tx=0x7f5b1c01c970 comp rx=0 tx=0).stop 2024-04-25T00:56:08.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- 172.21.15.112:0/2141327477 shutdown_connections 2024-04-25T00:56:08.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 --2- 172.21.15.112:0/2141327477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c0b9fc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 --2- 172.21.15.112:0/2141327477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c06c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- 172.21.15.112:0/2141327477 >> 172.21.15.112:0/2141327477 conn(0x7f5b2c066f50 msgr2=0x7f5b2c067330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- 172.21.15.112:0/2141327477 shutdown_connections 2024-04-25T00:56:08.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- 172.21.15.112:0/2141327477 wait complete. 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 Processor -- start 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- start start 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b2c0bae60 0x7f5b2c1e0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c1e3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c1549b0 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5b2c154830 con 0x7f5b2c0bae60 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b3259c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2c154b30 con 0x7f5b2c141a70 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60848/0 (socket says 172.21.15.112:60848) 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2bfff640 1 -- 172.21.15.112:0/3447115187 learned_addr learned my addr 172.21.15.112:0/3447115187 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b30b39640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c1e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2b7fe640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b2c0bae60 0x7f5b2c1e0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2bfff640 1 -- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 msgr2=0x7f5b2c1e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2bfff640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c1e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.783+0000 7f5b2bfff640 1 -- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b2c0bae60 msgr2=0x7f5b2c1e0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b2bfff640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b2c0bae60 0x7f5b2c1e0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b2bfff640 1 -- 172.21.15.112:0/3447115187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2c138680 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b30b39640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c1e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b2bfff640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c146490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c480 tx=0x7f5b2400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b28ff9640 1 -- 172.21.15.112:0/3447115187 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24017070 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b28ff9640 1 -- 172.21.15.112:0/3447115187 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b24005cf0 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2c1dfab0 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.788 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b28ff9640 1 -- 172.21.15.112:0/3447115187 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b24007290 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b2c151b70 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2c04f580 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b28ff9640 1 -- 172.21.15.112:0/3447115187 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5b2401a070 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.789 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b28ff9640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5b180678c0 0x7f5b18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b2b7fe640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5b180678c0 0x7f5b18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b28ff9640 1 -- 172.21.15.112:0/3447115187 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5b24085080 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.790 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.787+0000 7f5b2b7fe640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5b180678c0 0x7f5b18069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c06c4c0 tx=0x7f5b20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:08.792 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.791+0000 7f5b28ff9640 1 -- 172.21.15.112:0/3447115187 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5b24055c50 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:56:08.917 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:56:08.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.983+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5b2c06b9b0 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.985 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-04-25T00:56:08.983+0000 7f8589c5c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:56:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.987+0000 7f5b28ff9640 1 -- 172.21.15.112:0/3447115187 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f5b24055550 con 0x7f5b2c0b8e80 2024-04-25T00:56:08.987 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:56:08.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.987+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5b180678c0 msgr2=0x7f5b18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.987+0000 7f5b3259c640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5b180678c0 0x7f5b18069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c06c4c0 tx=0x7f5b20008040 comp rx=0 tx=0).stop 2024-04-25T00:56:08.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.987+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 msgr2=0x7f5b2c146490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:08.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.987+0000 7f5b3259c640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c146490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400c480 tx=0x7f5b2400c940 comp rx=0 tx=0).stop 2024-04-25T00:56:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.991+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 shutdown_connections 2024-04-25T00:56:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.991+0000 7f5b3259c640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5b2c141a70 0x7f5b2c1e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.991+0000 7f5b3259c640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5b180678c0 0x7f5b18069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.991+0000 7f5b3259c640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5b2c0bae60 0x7f5b2c1e0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.991+0000 7f5b3259c640 1 --2- 172.21.15.112:0/3447115187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b2c0b8e80 0x7f5b2c146490 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:08.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.991+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 >> 172.21.15.112:0/3447115187 conn(0x7f5b2c066f50 msgr2=0x7f5b2c13ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:08.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.991+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 shutdown_connections 2024-04-25T00:56:08.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:08.991+0000 7f5b3259c640 1 -- 172.21.15.112:0/3447115187 wait complete. 2024-04-25T00:56:08.998 INFO:tasks.ceph.mds.b.smithi187.stderr: -1127> 2024-04-25T00:55:44.052+0000 7f48af7b6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:56:09.007 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:56:09.051 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T00:56:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f664900e640 1 Processor -- start 2024-04-25T00:56:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f664900e640 1 -- start start 2024-04-25T00:56:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f664900e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 0x7f6644148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f664900e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66440b1390 con 0x7f66440affb0 2024-04-25T00:56:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f664900e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66440b1fa0 con 0x7f6644072b20 2024-04-25T00:56:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f664900e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f66440b1d70 con 0x7f66440b0390 2024-04-25T00:56:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6642d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f66440affb0 0x7f664409da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38840/0 (socket says 172.21.15.112:38840) 2024-04-25T00:56:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6642d9d640 1 -- 172.21.15.112:0/2914723303 learned_addr learned my addr 172.21.15.112:0/2914723303 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f664259c640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 0x7f6644148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4168485696 0 0) 0x7f66440b1390 con 0x7f66440affb0 2024-04-25T00:56:09.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f662c003530 con 0x7f66440affb0 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 603885644 0 0) 0x7f662c003530 con 0x7f66440affb0 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 msgr2=0x7f6644148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6641d9b640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 0x7f6644148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 >> v1:172.21.15.187:6789/0 conn(0x7f6644072b20 legacy=0x7f664414fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66440b34b0 con 0x7f66440affb0 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f663c001a00 con 0x7f66440affb0 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.167+0000 7f664259c640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 0x7f6644148800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 >> v1:172.21.15.112:6789/0 conn(0x7f66440affb0 legacy=0x7f664409da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 0x7f662c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c003fe0 0x7f662c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:09.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662c004d50 0x7f662c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f662c00bb90 con 0x7f662c004d50 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f662c00bd40 con 0x7f662c003fe0 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f662c00bf40 con 0x7f66440b0390 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664359e640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662c004d50 0x7f662c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6642d9d640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 0x7f662c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664259c640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c003fe0 0x7f662c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664359e640 1 -- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 msgr2=0x7f662c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664359e640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 0x7f662c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:09.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664359e640 1 -- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c003fe0 msgr2=0x7f662c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664359e640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f662c003fe0 0x7f662c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664359e640 1 -- 172.21.15.112:0/2914723303 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662c003530 con 0x7f662c004d50 2024-04-25T00:56:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6642d9d640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66440b0390 0x7f662c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664359e640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662c004d50 0x7f662c0051d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6634001780 tx=0x7f663401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3fed0d16c9d29a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340013a0 con 0x7f662c004d50 2024-04-25T00:56:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66340a9070 con 0x7f662c004d50 2024-04-25T00:56:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f6641d9b640 1 -- 172.21.15.112:0/2914723303 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634003600 con 0x7f662c004d50 2024-04-25T00:56:09.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664900e640 1 -- 172.21.15.112:0/2914723303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662c004d50 msgr2=0x7f662c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:09.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664900e640 1 --2- 172.21.15.112:0/2914723303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f662c004d50 0x7f662c0051d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6634001780 tx=0x7f663401c420 comp rx=0 tx=0).stop 2024-04-25T00:56:09.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664359e640 1 -- 172.21.15.112:0/2914723303 reap_dead start 2024-04-25T00:56:09.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664900e640 1 -- 172.21.15.112:0/2914723303 shutdown_connections 2024-04-25T00:56:09.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664900e640 1 -- 172.21.15.112:0/2914723303 >> 172.21.15.112:0/2914723303 conn(0x7f66440b8c80 msgr2=0x7f66440b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:09.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664900e640 1 -- 172.21.15.112:0/2914723303 shutdown_connections 2024-04-25T00:56:09.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.171+0000 7f664900e640 1 -- 172.21.15.112:0/2914723303 wait complete. 2024-04-25T00:56:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 Processor -- start 2024-04-25T00:56:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 -- start start 2024-04-25T00:56:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f664409b6a0 0x7f664409ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6644145ce0 0x7f664409c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:09.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664409c960 0x7f6644144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f66440b2520 con 0x7f664409c960 2024-04-25T00:56:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f66440b1600 con 0x7f664409b6a0 2024-04-25T00:56:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f664409e170 con 0x7f6644145ce0 2024-04-25T00:56:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664409c960 0x7f6644144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664409c960 0x7f6644144f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60862/0 (socket says 172.21.15.112:60862) 2024-04-25T00:56:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 -- 172.21.15.112:0/918787742 learned_addr learned my addr 172.21.15.112:0/918787742 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f6642d9d640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f664409b6a0 0x7f664409ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:09.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 -- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6644145ce0 msgr2=0x7f664409c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664259c640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6644145ce0 0x7f664409c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6644145ce0 0x7f664409c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 -- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f664409b6a0 msgr2=0x7f664409ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f664409b6a0 0x7f664409ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 -- 172.21.15.112:0/918787742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664409daa0 con 0x7f664409c960 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f6642d9d640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f664409b6a0 0x7f664409ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664359e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664409c960 0x7f6644144f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6634001780 tx=0x7f662c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f662b7fe640 1 -- 172.21.15.112:0/918787742 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340ba070 con 0x7f664409c960 2024-04-25T00:56:09.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f662b7fe640 1 -- 172.21.15.112:0/918787742 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6634000f60 con 0x7f664409c960 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66441db390 con 0x7f664409c960 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f662b7fe640 1 -- 172.21.15.112:0/918787742 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340013a0 con 0x7f664409c960 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66441db950 con 0x7f664409c960 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66441dbb50 con 0x7f664409c960 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f662b7fe640 1 -- 172.21.15.112:0/918787742 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f66340bd070 con 0x7f664409c960 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f662b7fe640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66300678f0 0x7f6630069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f662b7fe640 1 -- 172.21.15.112:0/918787742 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f663411ee60 con 0x7f664409c960 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.175+0000 7f6642d9d640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66300678f0 0x7f6630069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.179+0000 7f6642d9d640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66300678f0 0x7f6630069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f663c001590 tx=0x7f663c001840 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.179+0000 7f662b7fe640 1 -- 172.21.15.112:0/918787742 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f663414b5c0 con 0x7f664409c960 2024-04-25T00:56:09.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.375+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f66441db190 con 0x7f664409c960 2024-04-25T00:56:09.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.995+0000 7f662b7fe640 1 -- 172.21.15.112:0/918787742 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f66441dbb50 con 0x7f664409c960 2024-04-25T00:56:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66300678f0 msgr2=0x7f6630069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66300678f0 0x7f6630069db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f663c001590 tx=0x7f663c001840 comp rx=0 tx=0).stop 2024-04-25T00:56:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664409c960 msgr2=0x7f6644144f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664409c960 0x7f6644144f30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6634001780 tx=0x7f662c005840 comp rx=0 tx=0).stop 2024-04-25T00:56:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 shutdown_connections 2024-04-25T00:56:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f664409c960 0x7f6644144f30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.001 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6644145ce0 0x7f664409c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66300678f0 0x7f6630069db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 --2- 172.21.15.112:0/918787742 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f664409b6a0 0x7f664409ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 >> 172.21.15.112:0/918787742 conn(0x7f66440b8c80 msgr2=0x7f664413dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 shutdown_connections 2024-04-25T00:56:10.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:09.999+0000 7f664900e640 1 -- 172.21.15.112:0/918787742 wait complete. 2024-04-25T00:56:10.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:56:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00efb89640 1 Processor -- start 2024-04-25T00:56:10.151 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00efb89640 1 -- start start 2024-04-25T00:56:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00efb89640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f00e806e440 0x7f00e80af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00efb89640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e80b6380 con 0x7f00e8067b20 2024-04-25T00:56:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00efb89640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e809f480 con 0x7f00e809f0a0 2024-04-25T00:56:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00efb89640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f00e8067690 con 0x7f00e806e440 2024-04-25T00:56:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed124640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f00e8067b20 0x7f00e80b5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38848/0 (socket says 172.21.15.112:38848) 2024-04-25T00:56:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed124640 1 -- 172.21.15.112:0/3459972143 learned_addr learned my addr 172.21.15.112:0/3459972143 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:10.154 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ee126640 1 --2- 172.21.15.112:0/3459972143 >> v2:172.21.15.187:3300/0 conn(0x7f00e806e440 0x7f00e80af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3247798016 0 0) 0x7f00e80b6380 con 0x7f00e8067b20 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00d0003530 con 0x7f00e8067b20 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1715338501 0 0) 0x7f00d0003530 con 0x7f00e8067b20 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 >> v2:172.21.15.187:3300/0 conn(0x7f00e806e440 msgr2=0x7f00e80af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 --2- 172.21.15.112:0/3459972143 >> v2:172.21.15.187:3300/0 conn(0x7f00e806e440 0x7f00e80af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 >> v1:172.21.15.187:6789/0 conn(0x7f00e809f0a0 legacy=0x7f00e80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e80aec30 con 0x7f00e8067b20 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f00dc001680 con 0x7f00e8067b20 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 >> v1:172.21.15.112:6789/0 conn(0x7f00e8067b20 legacy=0x7f00e80b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d00039c0 0x7f00d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00d0004810 0x7f00d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00053d0 0x7f00d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8067690 con 0x7f00d00039c0 2024-04-25T00:56:10.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00d000c260 con 0x7f00d00053d0 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00d000c460 con 0x7f00d0004810 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed925640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d00039c0 0x7f00d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ee126640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00053d0 0x7f00d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed124640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00d0004810 0x7f00d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed925640 1 -- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00d0004810 msgr2=0x7f00d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed925640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00d0004810 0x7f00d0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed925640 1 -- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00053d0 msgr2=0x7f00d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed925640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00053d0 0x7f00d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ed925640 1 -- 172.21.15.112:0/3459972143 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d0003530 con 0x7f00d00039c0 2024-04-25T00:56:10.156 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ee126640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00053d0 0x7f00d000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:10.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.151+0000 7f00ee126640 1 -- 172.21.15.112:0/3459972143 reap_dead start 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00ed925640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d00039c0 0x7f00d0003de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f00e0001e70 tx=0x7f00e001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec26c7fc9b8fc8db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0001a90 con 0x7f00d00039c0 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00e0003b80 con 0x7f00d00039c0 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00ec923640 1 -- 172.21.15.112:0/3459972143 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0005080 con 0x7f00d00039c0 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 -- 172.21.15.112:0/3459972143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d00039c0 msgr2=0x7f00d0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d00039c0 0x7f00d0003de0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f00e0001e70 tx=0x7f00e001c730 comp rx=0 tx=0).stop 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 -- 172.21.15.112:0/3459972143 shutdown_connections 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 --2- 172.21.15.112:0/3459972143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00d00039c0 0x7f00d0003de0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 -- 172.21.15.112:0/3459972143 >> 172.21.15.112:0/3459972143 conn(0x7f00e80b6650 msgr2=0x7f00e80b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 -- 172.21.15.112:0/3459972143 shutdown_connections 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 -- 172.21.15.112:0/3459972143 wait complete. 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 Processor -- start 2024-04-25T00:56:10.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.155+0000 7f00efb89640 1 -- start start 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00039c0 0x7f00e8146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81dffa0 0x7f00e81e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e81e0420 0x7f00e81e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f00e806ebb0 con 0x7f00e81dffa0 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f00e80af7b0 con 0x7f00d00039c0 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f00e806ed30 con 0x7f00e81e0420 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81dffa0 0x7f00e81e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81dffa0 0x7f00e81e2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60884/0 (socket says 172.21.15.112:60884) 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 -- 172.21.15.112:0/4026388596 learned_addr learned my addr 172.21.15.112:0/4026388596 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed925640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00039c0 0x7f00e8146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ee126640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e81e0420 0x7f00e81e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 -- 172.21.15.112:0/4026388596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e81e0420 msgr2=0x7f00e81e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e81e0420 0x7f00e81e31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 -- 172.21.15.112:0/4026388596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00039c0 msgr2=0x7f00e8146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00039c0 0x7f00e8146f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 -- 172.21.15.112:0/4026388596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e80b4240 con 0x7f00e81dffa0 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed124640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81dffa0 0x7f00e81e2ca0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00d770 tx=0x7f00dc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00e67fc640 1 -- 172.21.15.112:0/4026388596 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc019070 con 0x7f00e81dffa0 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00e67fc640 1 -- 172.21.15.112:0/4026388596 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc0074e0 con 0x7f00e81dffa0 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81228f0 con 0x7f00e81dffa0 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00e67fc640 1 -- 172.21.15.112:0/4026388596 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0049f0 con 0x7f00e81dffa0 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e80b27b0 con 0x7f00e81dffa0 2024-04-25T00:56:10.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8068260 con 0x7f00e81dffa0 2024-04-25T00:56:10.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00e67fc640 1 -- 172.21.15.112:0/4026388596 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00dc01c070 con 0x7f00e81dffa0 2024-04-25T00:56:10.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00e67fc640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00d40678c0 0x7f00d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:10.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00e67fc640 1 -- 172.21.15.112:0/4026388596 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5795+0+0 (secure 0 0 0) 0x7f00dc0856a0 con 0x7f00e81dffa0 2024-04-25T00:56:10.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed925640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00d40678c0 0x7f00d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:10.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.159+0000 7f00ed925640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00d40678c0 0x7f00d4069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005000 tx=0x7f00e0001520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:10.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.163+0000 7f00e67fc640 1 -- 172.21.15.112:0/4026388596 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f00dc056170 con 0x7f00e81dffa0 2024-04-25T00:56:10.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.327+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f00e806c8b0 con 0x7f00e81dffa0 2024-04-25T00:56:10.999 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:10.999+0000 7f00e67fc640 1 -- 172.21.15.112:0/4026388596 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) ==== 181+0+0 (secure 0 0 0) 0x7f00dc055a70 con 0x7f00e81dffa0 2024-04-25T00:56:10.999 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:56:11.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00d40678c0 msgr2=0x7f00d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:11.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00d40678c0 0x7f00d4069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005000 tx=0x7f00e0001520 comp rx=0 tx=0).stop 2024-04-25T00:56:11.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81dffa0 msgr2=0x7f00e81e2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:11.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81dffa0 0x7f00e81e2ca0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00d770 tx=0x7f00dc00dc30 comp rx=0 tx=0).stop 2024-04-25T00:56:11.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 shutdown_connections 2024-04-25T00:56:11.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f00e81e0420 0x7f00e81e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f00e81dffa0 0x7f00e81e2ca0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f00d40678c0 0x7f00d4069d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 --2- 172.21.15.112:0/4026388596 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00d00039c0 0x7f00e8146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 >> 172.21.15.112:0/4026388596 conn(0x7f00e80b6650 msgr2=0x7f00e814ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:11.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 shutdown_connections 2024-04-25T00:56:11.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.003+0000 7f00efb89640 1 -- 172.21.15.112:0/4026388596 wait complete. 2024-04-25T00:56:11.025 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:56:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169f7b4640 1 Processor -- start 2024-04-25T00:56:11.155 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169f7b4640 1 -- start start 2024-04-25T00:56:11.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169f7b4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1698072520 0x7f16980b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:11.157 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169f7b4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16981419d0 0x7f1698151db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169f7b4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1698072900 con 0x7f169806ed10 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169f7b4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f16980b08e0 con 0x7f1698072520 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169f7b4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f169806ff60 con 0x7f16981419d0 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169d550640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1698072520 0x7f16980b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169d550640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1698072520 0x7f16980b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60896/0 (socket says 172.21.15.112:60896) 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169d550640 1 -- 172.21.15.112:0/4203251301 learned_addr learned my addr 172.21.15.112:0/4203251301 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169d550640 1 -- 172.21.15.112:0/4203251301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16981419d0 msgr2=0x7f1698151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169d550640 1 --2- 172.21.15.112:0/4203251301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16981419d0 0x7f1698151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169d550640 1 -- 172.21.15.112:0/4203251301 >> v1:172.21.15.187:6789/0 conn(0x7f169806ed10 legacy=0x7f16981412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:11.158 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.155+0000 7f169d550640 1 -- 172.21.15.112:0/4203251301 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169806fbf0 con 0x7f1698072520 2024-04-25T00:56:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169d550640 1 --2- 172.21.15.112:0/4203251301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698072520 0x7f16980b1180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1694009870 tx=0x7f169401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d32f3746351598e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f168ffff640 1 -- 172.21.15.112:0/4203251301 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16940b2070 con 0x7f1698072520 2024-04-25T00:56:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f168ffff640 1 -- 172.21.15.112:0/4203251301 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16940ad640 con 0x7f1698072520 2024-04-25T00:56:11.159 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f168ffff640 1 -- 172.21.15.112:0/4203251301 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16940ada10 con 0x7f1698072520 2024-04-25T00:56:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- 172.21.15.112:0/4203251301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698072520 msgr2=0x7f16980b1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 --2- 172.21.15.112:0/4203251301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698072520 0x7f16980b1180 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1694009870 tx=0x7f169401c520 comp rx=0 tx=0).stop 2024-04-25T00:56:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- 172.21.15.112:0/4203251301 shutdown_connections 2024-04-25T00:56:11.160 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 --2- 172.21.15.112:0/4203251301 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16981419d0 0x7f1698151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 --2- 172.21.15.112:0/4203251301 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1698072520 0x7f16980b1180 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- 172.21.15.112:0/4203251301 >> 172.21.15.112:0/4203251301 conn(0x7f16980baa70 msgr2=0x7f16980bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:11.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- 172.21.15.112:0/4203251301 shutdown_connections 2024-04-25T00:56:11.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- 172.21.15.112:0/4203251301 wait complete. 2024-04-25T00:56:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 Processor -- start 2024-04-25T00:56:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- start start 2024-04-25T00:56:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f169806ed10 0x7f169814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:11.162 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1698072520 0x7f1698146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16981419d0 0x7f169813a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16980b7370 con 0x7f16981419d0 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16980b71f0 con 0x7f169806ed10 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169f7b4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16980b0500 con 0x7f1698072520 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169dd51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16981419d0 0x7f169813a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169dd51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16981419d0 0x7f169813a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60908/0 (socket says 172.21.15.112:60908) 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.159+0000 7f169dd51640 1 -- 172.21.15.112:0/3154894937 learned_addr learned my addr 172.21.15.112:0/3154894937 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169cd4f640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1698072520 0x7f1698146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169dd51640 1 -- 172.21.15.112:0/3154894937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1698072520 msgr2=0x7f1698146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169dd51640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1698072520 0x7f1698146000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169dd51640 1 -- 172.21.15.112:0/3154894937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f169806ed10 msgr2=0x7f169814a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169dd51640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f169806ed10 0x7f169814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:11.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169dd51640 1 -- 172.21.15.112:0/3154894937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16981488c0 con 0x7f16981419d0 2024-04-25T00:56:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169cd4f640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1698072520 0x7f1698146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169dd51640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16981419d0 0x7f169813a240 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f169000b9a0 tx=0x7f169000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:11.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f168dffb640 1 -- 172.21.15.112:0/3154894937 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690019070 con 0x7f16981419d0 2024-04-25T00:56:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f168dffb640 1 -- 172.21.15.112:0/3154894937 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f169000cbc0 con 0x7f16981419d0 2024-04-25T00:56:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169f7b4640 1 -- 172.21.15.112:0/3154894937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169813b4b0 con 0x7f16981419d0 2024-04-25T00:56:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f168dffb640 1 -- 172.21.15.112:0/3154894937 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690014440 con 0x7f16981419d0 2024-04-25T00:56:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f169f7b4640 1 -- 172.21.15.112:0/3154894937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1698145b70 con 0x7f16981419d0 2024-04-25T00:56:11.165 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.163+0000 7f16837fe640 1 -- 172.21.15.112:0/3154894937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16700052a0 con 0x7f16981419d0 2024-04-25T00:56:11.166 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.167+0000 7f168dffb640 1 -- 172.21.15.112:0/3154894937 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f169001c070 con 0x7f16981419d0 2024-04-25T00:56:11.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.167+0000 7f168dffb640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f167c067940 0x7f167c069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:11.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.167+0000 7f168dffb640 1 -- 172.21.15.112:0/3154894937 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5357+0+0 (secure 0 0 0) 0x7f1690085750 con 0x7f16981419d0 2024-04-25T00:56:11.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.167+0000 7f169d550640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f167c067940 0x7f167c069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:11.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.167+0000 7f169d550640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f167c067940 0x7f167c069e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16940af000 tx=0x7f1694004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:11.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.171+0000 7f168dffb640 1 -- 172.21.15.112:0/3154894937 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1690056560 con 0x7f16981419d0 2024-04-25T00:56:11.330 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:11.331+0000 7f16837fe640 1 -- 172.21.15.112:0/3154894937 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1670005590 con 0x7f16981419d0 2024-04-25T00:56:12.007 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.007+0000 7f168dffb640 1 -- 172.21.15.112:0/3154894937 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) ==== 169+0+0 (secure 0 0 0) 0x7f1690055e60 con 0x7f16981419d0 2024-04-25T00:56:12.007 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-04-25T00:56:12.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.007+0000 7f16837fe640 1 -- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f167c067940 msgr2=0x7f167c069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:12.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.007+0000 7f16837fe640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f167c067940 0x7f167c069e00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f16940af000 tx=0x7f1694004740 comp rx=0 tx=0).stop 2024-04-25T00:56:12.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.007+0000 7f16837fe640 1 -- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16981419d0 msgr2=0x7f169813a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:12.010 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.007+0000 7f16837fe640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16981419d0 0x7f169813a240 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f169000b9a0 tx=0x7f169000be60 comp rx=0 tx=0).stop 2024-04-25T00:56:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.011+0000 7f16837fe640 1 -- 172.21.15.112:0/3154894937 shutdown_connections 2024-04-25T00:56:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.011+0000 7f16837fe640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16981419d0 0x7f169813a240 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.011+0000 7f16837fe640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1698072520 0x7f1698146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.011+0000 7f16837fe640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f167c067940 0x7f167c069e00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.011+0000 7f16837fe640 1 --2- 172.21.15.112:0/3154894937 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f169806ed10 0x7f169814a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.011+0000 7f16837fe640 1 -- 172.21.15.112:0/3154894937 >> 172.21.15.112:0/3154894937 conn(0x7f16980baa70 msgr2=0x7f169813d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.011+0000 7f16837fe640 1 -- 172.21.15.112:0/3154894937 shutdown_connections 2024-04-25T00:56:12.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.011+0000 7f16837fe640 1 -- 172.21.15.112:0/3154894937 wait complete. 2024-04-25T00:56:12.038 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T00:56:12.038 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T00:56:12.038 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:56:12.038 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:56:12.038 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T00:56:12.038 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T00:56:12.039 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:56:12.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:56:12.048 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:56:12.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T00:56:12.052 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:56:12.052 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T00:56:12.091 INFO:tasks.ceph.mds.a:Started 2024-04-25T00:56:12.092 DEBUG:teuthology.parallel:result is None 2024-04-25T00:56:12.094 INFO:tasks.ceph.mds.b:Started 2024-04-25T00:56:12.094 DEBUG:teuthology.parallel:result is None 2024-04-25T00:56:12.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T00:56:12.159 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-04-25T00:56:12.163 INFO:tasks.ceph.mds.b.smithi187.stdout:starting mds.b at 2024-04-25T00:56:12.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd9462640 1 Processor -- start 2024-04-25T00:56:12.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd9462640 1 -- start start 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd9462640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4bd40add70 0x7f4bd4145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd9462640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd40aeec0 con 0x7f4bd40b23f0 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd9462640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd40af780 con 0x7f4bd40b3ce0 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd9462640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd40b27d0 con 0x7f4bd40add70 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd2ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f4bd40b23f0 0x7f4bd4072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:38854/0 (socket says 172.21.15.112:38854) 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd2ffd640 1 -- 172.21.15.112:0/3698720877 learned_addr learned my addr 172.21.15.112:0/3698720877 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd27fc640 1 --2- 172.21.15.112:0/3698720877 >> v2:172.21.15.112:3300/0 conn(0x7f4bd40add70 0x7f4bd4145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd1ffb640 1 -- 172.21.15.112:0/3698720877 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731881343 0 0) 0x7f4bd40aeec0 con 0x7f4bd40b23f0 2024-04-25T00:56:12.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd1ffb640 1 -- 172.21.15.112:0/3698720877 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bbc003530 con 0x7f4bd40b23f0 2024-04-25T00:56:12.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd27fc640 1 -- 172.21.15.112:0/3698720877 >> v1:172.21.15.187:6789/0 conn(0x7f4bd40b3ce0 legacy=0x7f4bd414fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:12.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd27fc640 1 -- 172.21.15.112:0/3698720877 >> v1:172.21.15.112:6789/0 conn(0x7f4bd40b23f0 legacy=0x7f4bd4072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:12.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd27fc640 1 -- 172.21.15.112:0/3698720877 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd40afa20 con 0x7f4bd40add70 2024-04-25T00:56:12.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.215+0000 7f4bd27fc640 1 --2- 172.21.15.112:0/3698720877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4145200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40af950 tx=0x7f4bc801cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c8f95be3b3e9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:12.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd1ffb640 1 -- 172.21.15.112:0/3698720877 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80aa620 con 0x7f4bd40add70 2024-04-25T00:56:12.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd1ffb640 1 -- 172.21.15.112:0/3698720877 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc80af070 con 0x7f4bd40add70 2024-04-25T00:56:12.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd1ffb640 1 -- 172.21.15.112:0/3698720877 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80aa9e0 con 0x7f4bd40add70 2024-04-25T00:56:12.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- 172.21.15.112:0/3698720877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 msgr2=0x7f4bd4145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:12.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 --2- 172.21.15.112:0/3698720877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4145200 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40af950 tx=0x7f4bc801cf00 comp rx=0 tx=0).stop 2024-04-25T00:56:12.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- 172.21.15.112:0/3698720877 shutdown_connections 2024-04-25T00:56:12.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 --2- 172.21.15.112:0/3698720877 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4145200 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:12.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- 172.21.15.112:0/3698720877 >> 172.21.15.112:0/3698720877 conn(0x7f4bd40b5980 msgr2=0x7f4bd40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:12.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- 172.21.15.112:0/3698720877 shutdown_connections 2024-04-25T00:56:12.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- 172.21.15.112:0/3698720877 wait complete. 2024-04-25T00:56:12.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 Processor -- start 2024-04-25T00:56:12.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- start start 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4140880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd40b23f0 0x7f4bd40713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd40b3ce0 0x7f4bd41e1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd40b52d0 con 0x7f4bd40add70 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd40b5150 con 0x7f4bd40b3ce0 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd4153a80 con 0x7f4bd40b23f0 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4140880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60930/0 (socket says 172.21.15.112:60930) 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 -- 172.21.15.112:0/899855008 learned_addr learned my addr 172.21.15.112:0/899855008 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd37fe640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd40b3ce0 0x7f4bd41e1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd27fc640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd40b23f0 0x7f4bd40713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 -- 172.21.15.112:0/899855008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd40b23f0 msgr2=0x7f4bd40713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd40b23f0 0x7f4bd40713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:12.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 -- 172.21.15.112:0/899855008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd40b3ce0 msgr2=0x7f4bd41e1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd40b3ce0 0x7f4bd41e1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 -- 172.21.15.112:0/899855008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd4140fd0 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd2ffd640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4140880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400c4d0 tx=0x7f4bc400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bbb7fe640 1 -- 172.21.15.112:0/899855008 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4017070 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bbb7fe640 1 -- 172.21.15.112:0/899855008 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4005cf0 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd41412b0 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bbb7fe640 1 -- 172.21.15.112:0/899855008 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4007270 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.219+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd4140050 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.223+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd404f580 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.223+0000 7f4bbb7fe640 1 -- 172.21.15.112:0/899855008 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4bc401a070 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.223+0000 7f4bbb7fe640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc00678c0 0x7f4bc0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.223+0000 7f4bbb7fe640 1 -- 172.21.15.112:0/899855008 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 4943+0+0 (secure 0 0 0) 0x7f4bc4084d00 con 0x7f4bd40add70 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.223+0000 7f4bd27fc640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc00678c0 0x7f4bc0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:12.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.223+0000 7f4bd27fc640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc00678c0 0x7f4bc0069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ae000 tx=0x7f4bc8004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:12.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.227+0000 7f4bbb7fe640 1 -- 172.21.15.112:0/899855008 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4bc4055500 con 0x7f4bd40add70 2024-04-25T00:56:12.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:12.411+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4bd409f0c0 con 0x7f4bd40add70 2024-04-25T00:56:13.025 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.023+0000 7f4bbb7fe640 1 -- 172.21.15.112:0/899855008 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) ==== 97+0+0 (secure 0 0 0) 0x7f4bc4059800 con 0x7f4bd40add70 2024-04-25T00:56:13.025 INFO:teuthology.orchestra.run.smithi112.stderr: removed all blocklist entries 2024-04-25T00:56:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc00678c0 msgr2=0x7f4bc0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc00678c0 0x7f4bc0069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ae000 tx=0x7f4bc8004740 comp rx=0 tx=0).stop 2024-04-25T00:56:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 msgr2=0x7f4bd4140880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4140880 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4bc400c4d0 tx=0x7f4bc400c990 comp rx=0 tx=0).stop 2024-04-25T00:56:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 shutdown_connections 2024-04-25T00:56:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f4bc00678c0 0x7f4bc0069d80 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4bc80ae000 tx=0x7f4bc8004740 comp rx=0 tx=0).stop 2024-04-25T00:56:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f4bd40b3ce0 0x7f4bd41e1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f4bd40b23f0 0x7f4bd40713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 --2- 172.21.15.112:0/899855008 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4bd40add70 0x7f4bd4140880 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 >> 172.21.15.112:0/899855008 conn(0x7f4bd40b5980 msgr2=0x7f4bd40b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 shutdown_connections 2024-04-25T00:56:13.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.027+0000 7f4bd9462640 1 -- 172.21.15.112:0/899855008 wait complete. 2024-04-25T00:56:13.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T00:56:13.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.171+0000 7f6eda232640 1 Processor -- start 2024-04-25T00:56:13.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.171+0000 7f6eda232640 1 -- start start 2024-04-25T00:56:13.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.171+0000 7f6eda232640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6ed40adfd0 0x7f6ed40ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.171+0000 7f6eda232640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ed40ae9a0 0x7f6ed40b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.171+0000 7f6eda232640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40aee30 0x7f6ed40b2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.175 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed40b0800 con 0x7f6ed40ae9a0 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed40b52f0 con 0x7f6ed40adfd0 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed4143f50 con 0x7f6ed40aee30 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ed40ae9a0 0x7f6ed40b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6ed40ae9a0 0x7f6ed40b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33490/0 (socket says 172.21.15.112:33490) 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 -- 172.21.15.112:0/353128797 learned_addr learned my addr 172.21.15.112:0/353128797 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed37fe640 1 --2- 172.21.15.112:0/353128797 >> v2:172.21.15.187:3300/0 conn(0x7f6ed40adfd0 0x7f6ed40ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed3fff640 1 --2- 172.21.15.112:0/353128797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40aee30 0x7f6ed40b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 -- 172.21.15.112:0/353128797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40aee30 msgr2=0x7f6ed40b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 --2- 172.21.15.112:0/353128797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40aee30 0x7f6ed40b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 -- 172.21.15.112:0/353128797 >> v2:172.21.15.187:3300/0 conn(0x7f6ed40adfd0 msgr2=0x7f6ed40ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:13.176 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 --2- 172.21.15.112:0/353128797 >> v2:172.21.15.187:3300/0 conn(0x7f6ed40adfd0 0x7f6ed40ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 -- 172.21.15.112:0/353128797 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed4144230 con 0x7f6ed40ae9a0 2024-04-25T00:56:13.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed37fe640 1 --2- 172.21.15.112:0/353128797 >> v2:172.21.15.187:3300/0 conn(0x7f6ed40adfd0 0x7f6ed40ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:13.177 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed3fff640 1 --2- 172.21.15.112:0/353128797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40aee30 0x7f6ed40b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:13.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed2ffd640 1 --2- 172.21.15.112:0/353128797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40ae9a0 0x7f6ed40b2240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40b4ea0 tx=0x7f6ec801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531f8b29ab55d25b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed27fc640 1 -- 172.21.15.112:0/353128797 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80b0070 con 0x7f6ed40ae9a0 2024-04-25T00:56:13.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed27fc640 1 -- 172.21.15.112:0/353128797 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec80ab600 con 0x7f6ed40ae9a0 2024-04-25T00:56:13.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6ed27fc640 1 -- 172.21.15.112:0/353128797 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80ab9d0 con 0x7f6ed40ae9a0 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 -- 172.21.15.112:0/353128797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40ae9a0 msgr2=0x7f6ed40b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 --2- 172.21.15.112:0/353128797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40ae9a0 0x7f6ed40b2240 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40b4ea0 tx=0x7f6ec801c490 comp rx=0 tx=0).stop 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 -- 172.21.15.112:0/353128797 shutdown_connections 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 --2- 172.21.15.112:0/353128797 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40aee30 0x7f6ed40b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 --2- 172.21.15.112:0/353128797 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40ae9a0 0x7f6ed40b2240 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 --2- 172.21.15.112:0/353128797 >> v2:172.21.15.187:3300/0 conn(0x7f6ed40adfd0 0x7f6ed40ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 -- 172.21.15.112:0/353128797 >> 172.21.15.112:0/353128797 conn(0x7f6ed40b5980 msgr2=0x7f6ed40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 -- 172.21.15.112:0/353128797 shutdown_connections 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.175+0000 7f6eda232640 1 -- 172.21.15.112:0/353128797 wait complete. 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 Processor -- start 2024-04-25T00:56:13.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 -- start start 2024-04-25T00:56:13.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed40adfd0 0x7f6ed41342a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40ae9a0 0x7f6ed41e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40aee30 0x7f6ed41323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4147270 con 0x7f6ed40aee30 2024-04-25T00:56:13.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41470f0 con 0x7f6ed40adfd0 2024-04-25T00:56:13.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed41473f0 con 0x7f6ed40ae9a0 2024-04-25T00:56:13.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40aee30 0x7f6ed41323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40aee30 0x7f6ed41323e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33492/0 (socket says 172.21.15.112:33492) 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 -- 172.21.15.112:0/1407681700 learned_addr learned my addr 172.21.15.112:0/1407681700 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed37fe640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed40adfd0 0x7f6ed41342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed2ffd640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40ae9a0 0x7f6ed41e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 -- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40ae9a0 msgr2=0x7f6ed41e02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40ae9a0 0x7f6ed41e02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 -- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed40adfd0 msgr2=0x7f6ed41342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed40adfd0 0x7f6ed41342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 -- 172.21.15.112:0/1407681700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed4130db0 con 0x7f6ed40aee30 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed2ffd640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40ae9a0 0x7f6ed41e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed37fe640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed40adfd0 0x7f6ed41342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ed3fff640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40aee30 0x7f6ed41323e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00cc90 tx=0x7f6ecc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ebbfff640 1 -- 172.21.15.112:0/1407681700 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc003ba0 con 0x7f6ed40aee30 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ebbfff640 1 -- 172.21.15.112:0/1407681700 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ecc003d60 con 0x7f6ed40aee30 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed4130ad0 con 0x7f6ed40aee30 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ebbfff640 1 -- 172.21.15.112:0/1407681700 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc004260 con 0x7f6ed40aee30 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed41dfa50 con 0x7f6ed40aee30 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed404f580 con 0x7f6ed40aee30 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ebbfff640 1 -- 172.21.15.112:0/1407681700 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6ecc004400 con 0x7f6ed40aee30 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.179+0000 7f6ebbfff640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ec00678c0 0x7f6ec0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.183+0000 7f6ebbfff640 1 -- 172.21.15.112:0/1407681700 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f6ecc086190 con 0x7f6ed40aee30 2024-04-25T00:56:13.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.183+0000 7f6ed37fe640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ec00678c0 0x7f6ec0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.185 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.183+0000 7f6ed37fe640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ec00678c0 0x7f6ec0069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4006fd0 tx=0x7f6ec4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.186 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.187+0000 7f6ebbfff640 1 -- 172.21.15.112:0/1407681700 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6ed4130db0 con 0x7f6ed40aee30 2024-04-25T00:56:13.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.403+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6ed413ee90 con 0x7f6ed40aee30 2024-04-25T00:56:13.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.403+0000 7f6ebbfff640 1 -- 172.21.15.112:0/1407681700 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+5207 (secure 0 0 0) 0x7f6ecc014030 con 0x7f6ed40aee30 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "auth_dump": [ 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.a", 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDRqSlmB1p7AxAApj8b+uJ+23NHlPP7yC8SJg==", 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.405 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mds.b", 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDRqSlmgYqACRAA9nOmbvfzq2/LHofYEsUIyQ==", 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.0", 2024-04-25T00:56:13.406 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDUqSlmNBbtEBAAJgEN6Jqa3Y70/wPnOMJcqA==", 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.1", 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDXqSlm8n6SFRAAQZXRwyE8uvokp9/lel0B9w==", 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.407 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.2", 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDaqSlmnbgHHhAA3pEOhNODws3ADApmcA02Dw==", 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.3", 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDdqSlmXGnOJBAAdlkLgvu931KjX6vM1rUN+A==", 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.408 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.4", 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDjqSlmxA0mIBAASNsLaxsy+z4/T5zBzV6//Q==", 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.409 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.5", 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDmqSlmH7SmJRAAj15G2CcndsUB+HV0OCS6kw==", 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.6", 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDqqSlmr+i6BxAARbD2r40Zydtd1IkdC8RTVg==", 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-04-25T00:56:13.410 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "osd.7", 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDtqSlmcTMBDBAAIgIUHgsu/mBiq8QNBDDppw==", 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow profile osd", 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile osd", 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.0", 2024-04-25T00:56:13.411 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDRqSlmLHGXDBAAcY5ixa+8Gv2MeY7OW7Jffw==", 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow", 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow r", 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow rw", 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow rwx" 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.admin", 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDQqSlm4VhyKBAAB8BqMRL6B98a5wap3cA3Rg==", 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "mgr": "allow *", 2024-04-25T00:56:13.412 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow *", 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-mds", 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDyqSlmHks8NBAADcvQQTcL2Qd8HgFrSh439A==", 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDyqSlm0mU8NBAAse8RTo4DjT8yETvL34d3MA==", 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.413 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-osd", 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDyqSlmc348NBAAwdjTPkAy0FjERXvi3oAP+w==", 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDyqSlm7ZQ8NBAAc74C2tbxFww8qfzm4p7XwQ==", 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.414 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDyqSlm2qs8NBAAFEgce45EFP8Lz3G0/Foiag==", 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDyqSlmu8M8NBAAFWwvu77WSMjFv4nR7FRQLw==", 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.415 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.x", 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDRqSlmviQvABAARqSTcVrjv+0YqfhTPI4UMg==", 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "entity": "mgr.y", 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "key": "AQDQqSlm0iw+ORAAmDIlUnXiJ58JXA14Mt1Qpg==", 2024-04-25T00:56:13.416 INFO:teuthology.orchestra.run.smithi112.stdout: "caps": { 2024-04-25T00:56:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "mds": "allow *", 2024-04-25T00:56:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "mon": "allow profile mgr", 2024-04-25T00:56:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: "osd": "allow *" 2024-04-25T00:56:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:13.417 INFO:teuthology.orchestra.run.smithi112.stdout: ] 2024-04-25T00:56:13.417 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ec00678c0 msgr2=0x7f6ec0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ec00678c0 0x7f6ec0069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4006fd0 tx=0x7f6ec4008040 comp rx=0 tx=0).stop 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40aee30 msgr2=0x7f6ed41323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40aee30 0x7f6ed41323e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00cc90 tx=0x7f6ecc00a650 comp rx=0 tx=0).stop 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 shutdown_connections 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6ed40aee30 0x7f6ed41323e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6ed40ae9a0 0x7f6ed41e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ec00678c0 0x7f6ec0069d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 --2- 172.21.15.112:0/1407681700 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6ed40adfd0 0x7f6ed41342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 >> 172.21.15.112:0/1407681700 conn(0x7f6ed40b5980 msgr2=0x7f6ed41501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:13.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 shutdown_connections 2024-04-25T00:56:13.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.407+0000 7f6eda232640 1 -- 172.21.15.112:0/1407681700 wait complete. 2024-04-25T00:56:13.423 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T00:56:13.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1ecb49640 1 Processor -- start 2024-04-25T00:56:13.539 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1ecb49640 1 -- start start 2024-04-25T00:56:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1ecb49640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fb1e806c630 0x7fb1e80aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1ecb49640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb1e806ca10 0x7fb1e80b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1ecb49640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e80add70 0x7fb1e81477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1ecb49640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e8148860 con 0x7fb1e806ca10 2024-04-25T00:56:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1ecb49640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e80b52f0 con 0x7fb1e806c630 2024-04-25T00:56:13.540 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1ecb49640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e80b1060 con 0x7fb1e80add70 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb1e806ca10 0x7fb1e80b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb1e806ca10 0x7fb1e80b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33506/0 (socket says 172.21.15.112:33506) 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 -- 172.21.15.112:0/3101026706 learned_addr learned my addr 172.21.15.112:0/3101026706 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1e659c640 1 --2- 172.21.15.112:0/3101026706 >> v2:172.21.15.187:3300/0 conn(0x7fb1e806c630 0x7fb1e80aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 -- 172.21.15.112:0/3101026706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e80add70 msgr2=0x7fb1e81477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1e6d9d640 1 --2- 172.21.15.112:0/3101026706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e80add70 0x7fb1e81477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 --2- 172.21.15.112:0/3101026706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e80add70 0x7fb1e81477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 -- 172.21.15.112:0/3101026706 >> v2:172.21.15.187:3300/0 conn(0x7fb1e806c630 msgr2=0x7fb1e80aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 --2- 172.21.15.112:0/3101026706 >> v2:172.21.15.187:3300/0 conn(0x7fb1e806c630 0x7fb1e80aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.541 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 -- 172.21.15.112:0/3101026706 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e80af760 con 0x7fb1e806ca10 2024-04-25T00:56:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.539+0000 7fb1dffff640 1 --2- 172.21.15.112:0/3101026706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806ca10 0x7fb1e80b4db0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80b9bc0 tx=0x7fb1d001c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95638b3e1f095eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e5d9b640 1 -- 172.21.15.112:0/3101026706 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00b0070 con 0x7fb1e806ca10 2024-04-25T00:56:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e5d9b640 1 -- 172.21.15.112:0/3101026706 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d00ab600 con 0x7fb1e806ca10 2024-04-25T00:56:13.542 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e5d9b640 1 -- 172.21.15.112:0/3101026706 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00ab9d0 con 0x7fb1e806ca10 2024-04-25T00:56:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- 172.21.15.112:0/3101026706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806ca10 msgr2=0x7fb1e80b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/3101026706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806ca10 0x7fb1e80b4db0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb1e80b9bc0 tx=0x7fb1d001c140 comp rx=0 tx=0).stop 2024-04-25T00:56:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- 172.21.15.112:0/3101026706 shutdown_connections 2024-04-25T00:56:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/3101026706 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e80add70 0x7fb1e81477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/3101026706 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806ca10 0x7fb1e80b4db0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.543 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/3101026706 >> v2:172.21.15.187:3300/0 conn(0x7fb1e806c630 0x7fb1e80aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- 172.21.15.112:0/3101026706 >> 172.21.15.112:0/3101026706 conn(0x7fb1e80b5980 msgr2=0x7fb1e80b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- 172.21.15.112:0/3101026706 shutdown_connections 2024-04-25T00:56:13.544 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- 172.21.15.112:0/3101026706 wait complete. 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 Processor -- start 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- start start 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806c630 0x7fb1e809e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e806ca10 0x7fb1e809e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e80add70 0x7fb1e809aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b2370 con 0x7fb1e806c630 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80b21f0 con 0x7fb1e80add70 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e80b24f0 con 0x7fb1e806ca10 2024-04-25T00:56:13.546 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806c630 0x7fb1e809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806c630 0x7fb1e809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33520/0 (socket says 172.21.15.112:33520) 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 -- 172.21.15.112:0/71501174 learned_addr learned my addr 172.21.15.112:0/71501174 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1dffff640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e806ca10 0x7fb1e809e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e6d9d640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e80add70 0x7fb1e809aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 -- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e806ca10 msgr2=0x7fb1e809e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e806ca10 0x7fb1e809e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 -- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e80add70 msgr2=0x7fb1e809aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e80add70 0x7fb1e809aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 -- 172.21.15.112:0/71501174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e80b3170 con 0x7fb1e806c630 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1dffff640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e806ca10 0x7fb1e809e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1e659c640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806c630 0x7fb1e809e220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800d800 tx=0x7fb1d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1deffd640 1 -- 172.21.15.112:0/71501174 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8018020 con 0x7fb1e806c630 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1deffd640 1 -- 172.21.15.112:0/71501174 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d80096a0 con 0x7fb1e806c630 2024-04-25T00:56:13.547 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e809caf0 con 0x7fb1e806c630 2024-04-25T00:56:13.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1deffd640 1 -- 172.21.15.112:0/71501174 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8009a50 con 0x7fb1e806c630 2024-04-25T00:56:13.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.543+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e809c810 con 0x7fb1e806c630 2024-04-25T00:56:13.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.547+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e804f580 con 0x7fb1e806c630 2024-04-25T00:56:13.548 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.547+0000 7fb1deffd640 1 -- 172.21.15.112:0/71501174 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb1d801b020 con 0x7fb1e806c630 2024-04-25T00:56:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.547+0000 7fb1deffd640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb1cc0678c0 0x7fb1cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.547+0000 7fb1deffd640 1 -- 172.21.15.112:0/71501174 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fb1d8085e20 con 0x7fb1e806c630 2024-04-25T00:56:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.547+0000 7fb1dffff640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb1cc0678c0 0x7fb1cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.549 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.547+0000 7fb1dffff640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb1cc0678c0 0x7fb1cc069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00ad000 tx=0x7fb1d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.551+0000 7fb1deffd640 1 -- 172.21.15.112:0/71501174 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb1d8022020 con 0x7fb1e806c630 2024-04-25T00:56:13.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.751+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fb1e8141b20 con 0x7fb1e806c630 2024-04-25T00:56:13.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1deffd640 1 -- 172.21.15.112:0/71501174 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fb1e809caf0 con 0x7fb1e806c630 2024-04-25T00:56:13.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb1cc0678c0 msgr2=0x7fb1cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb1cc0678c0 0x7fb1cc069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00ad000 tx=0x7fb1d0008040 comp rx=0 tx=0).stop 2024-04-25T00:56:13.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806c630 msgr2=0x7fb1e809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806c630 0x7fb1e809e220 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800d800 tx=0x7fb1d800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 shutdown_connections 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb1cc0678c0 0x7fb1cc069d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb1e80add70 0x7fb1e809aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb1e806ca10 0x7fb1e809e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 --2- 172.21.15.112:0/71501174 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1e806c630 0x7fb1e809e220 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 >> 172.21.15.112:0/71501174 conn(0x7fb1e80b5980 msgr2=0x7fb1e80b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 shutdown_connections 2024-04-25T00:56:13.758 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.755+0000 7fb1ecb49640 1 -- 172.21.15.112:0/71501174 wait complete. 2024-04-25T00:56:13.774 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T00:56:13.892 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7f0004942640 1 Processor -- start 2024-04-25T00:56:13.893 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7f0004942640 1 -- start start 2024-04-25T00:56:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7f0004942640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f000009c270 0x7f0000147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7f0004942640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00000bbec0 con 0x7f00000722d0 2024-04-25T00:56:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7f0004942640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00000bc5c0 con 0x7f0000147650 2024-04-25T00:56:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7f0004942640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f00000baaf0 con 0x7f000009c270 2024-04-25T00:56:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7efffe59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f00000722d0 0x7f0000072bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41256/0 (socket says 172.21.15.112:41256) 2024-04-25T00:56:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7efffe59c640 1 -- 172.21.15.112:0/730689968 learned_addr learned my addr 172.21.15.112:0/730689968 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:13.894 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7efff5d9b640 1 --2- 172.21.15.112:0/730689968 >> v2:172.21.15.112:3300/0 conn(0x7f000009c270 0x7f0000147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7efffdd9b640 1 -- 172.21.15.112:0/730689968 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2882752464 0 0) 0x7f00000bbec0 con 0x7f00000722d0 2024-04-25T00:56:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.891+0000 7efffdd9b640 1 -- 172.21.15.112:0/730689968 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effec003530 con 0x7f00000722d0 2024-04-25T00:56:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7efff5d9b640 1 -- 172.21.15.112:0/730689968 >> v1:172.21.15.187:6789/0 conn(0x7f0000147650 legacy=0x7f0000151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7efff5d9b640 1 -- 172.21.15.112:0/730689968 >> v1:172.21.15.112:6789/0 conn(0x7f00000722d0 legacy=0x7f0000072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7efffdd9b640 1 -- 172.21.15.112:0/730689968 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4010660516 0 0) 0x7effec003530 con 0x7f00000722d0 2024-04-25T00:56:13.895 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7efff5d9b640 1 -- 172.21.15.112:0/730689968 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00000bc8a0 con 0x7f000009c270 2024-04-25T00:56:13.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7efff5d9b640 1 --2- 172.21.15.112:0/730689968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f0000147d00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effe80097c0 tx=0x7effe801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3936adb9863e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7efffdd9b640 1 -- 172.21.15.112:0/730689968 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe80b1070 con 0x7f000009c270 2024-04-25T00:56:13.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7efffdd9b640 1 -- 172.21.15.112:0/730689968 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe80ac440 con 0x7f000009c270 2024-04-25T00:56:13.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7efffdd9b640 1 -- 172.21.15.112:0/730689968 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff00016a0 con 0x7f000009c270 2024-04-25T00:56:13.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7f0004942640 1 -- 172.21.15.112:0/730689968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 msgr2=0x7f0000147d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7f0004942640 1 --2- 172.21.15.112:0/730689968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f0000147d00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effe80097c0 tx=0x7effe801c390 comp rx=0 tx=0).stop 2024-04-25T00:56:13.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7f0004942640 1 -- 172.21.15.112:0/730689968 shutdown_connections 2024-04-25T00:56:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7f0004942640 1 --2- 172.21.15.112:0/730689968 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f0000147d00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7f0004942640 1 -- 172.21.15.112:0/730689968 >> 172.21.15.112:0/730689968 conn(0x7f000009e8c0 msgr2=0x7f000009ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7f0004942640 1 -- 172.21.15.112:0/730689968 shutdown_connections 2024-04-25T00:56:13.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.895+0000 7f0004942640 1 -- 172.21.15.112:0/730689968 wait complete. 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 Processor -- start 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 -- start start 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00000722d0 0x7f000013e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f000014a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0000147650 0x7f000014a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0000155fb0 con 0x7f000009c270 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0000155e30 con 0x7f00000722d0 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0000156130 con 0x7f0000147650 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f000014a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f000014a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33532/0 (socket says 172.21.15.112:33532) 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 -- 172.21.15.112:0/1873570679 learned_addr learned my addr 172.21.15.112:0/1873570679 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efffe59c640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00000722d0 0x7f000013e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 -- 172.21.15.112:0/1873570679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0000147650 msgr2=0x7f000014a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0000147650 0x7f000014a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 -- 172.21.15.112:0/1873570679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00000722d0 msgr2=0x7f000013e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00000722d0 0x7f000013e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 -- 172.21.15.112:0/1873570679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0000149cd0 con 0x7f000009c270 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff5d9b640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f000014a380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7effe80af040 tx=0x7effe801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff77fe640 1 -- 172.21.15.112:0/1873570679 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe80bc020 con 0x7f000009c270 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff77fe640 1 -- 172.21.15.112:0/1873570679 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effe80b2040 con 0x7f000009c270 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00000b1100 con 0x7f000009c270 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff77fe640 1 -- 172.21.15.112:0/1873570679 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe80b1050 con 0x7f000009c270 2024-04-25T00:56:13.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00001487b0 con 0x7f000009c270 2024-04-25T00:56:13.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0000149410 con 0x7f000009c270 2024-04-25T00:56:13.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff77fe640 1 -- 172.21.15.112:0/1873570679 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7effe80bf070 con 0x7f000009c270 2024-04-25T00:56:13.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff77fe640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7effe00678f0 0x7effe0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:13.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efffe59c640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7effe00678f0 0x7effe0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:13.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.899+0000 7efff77fe640 1 -- 172.21.15.112:0/1873570679 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7effe811f140 con 0x7f000009c270 2024-04-25T00:56:13.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.903+0000 7efffe59c640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7effe00678f0 0x7effe0069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efff0001ec0 tx=0x7efff0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:13.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:13.907+0000 7efff77fe640 1 -- 172.21.15.112:0/1873570679 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7effe8015fb0 con 0x7f000009c270 2024-04-25T00:56:14.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.119+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f000004f580 con 0x7f000009c270 2024-04-25T00:56:14.124 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.123+0000 7efff77fe640 1 -- 172.21.15.112:0/1873570679 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7effe80ab3a0 con 0x7f000009c270 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7effe00678f0 msgr2=0x7effe0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7effe00678f0 0x7effe0069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efff0001ec0 tx=0x7efff0009040 comp rx=0 tx=0).stop 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 msgr2=0x7f000014a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f000014a380 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7effe80af040 tx=0x7effe801abd0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 shutdown_connections 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0000147650 0x7f000014a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f000009c270 0x7f000014a380 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7effe00678f0 0x7effe0069db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 --2- 172.21.15.112:0/1873570679 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f00000722d0 0x7f000013e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 >> 172.21.15.112:0/1873570679 conn(0x7f000009e8c0 msgr2=0x7f00000bd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:14.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 shutdown_connections 2024-04-25T00:56:14.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.127+0000 7f0004942640 1 -- 172.21.15.112:0/1873570679 wait complete. 2024-04-25T00:56:14.142 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T00:56:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b8dbd640 1 Processor -- start 2024-04-25T00:56:14.250 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b8dbd640 1 -- start start 2024-04-25T00:56:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b8dbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b40b23f0 0x7fc1b40add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b8dbd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1b4145200 0x7fc1b40b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b8dbd640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1b40aeec0 con 0x7fc1b4072760 2024-04-25T00:56:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b8dbd640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc1b40af780 con 0x7fc1b4145200 2024-04-25T00:56:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b8dbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b40b27d0 con 0x7fc1b40b23f0 2024-04-25T00:56:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1b4145200 0x7fc1b40b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1abfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc1b4145200 0x7fc1b40b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33544/0 (socket says 172.21.15.112:33544) 2024-04-25T00:56:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1abfff640 1 -- 172.21.15.112:0/3426278345 learned_addr learned my addr 172.21.15.112:0/3426278345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b259c640 1 --2- 172.21.15.112:0/3426278345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b40b23f0 0x7fc1b40add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1abfff640 1 -- 172.21.15.112:0/3426278345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b40b23f0 msgr2=0x7fc1b40add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1abfff640 1 --2- 172.21.15.112:0/3426278345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b40b23f0 0x7fc1b40add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1abfff640 1 -- 172.21.15.112:0/3426278345 >> v1:172.21.15.187:6789/0 conn(0x7fc1b4072760 legacy=0x7fc1b406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1abfff640 1 -- 172.21.15.112:0/3426278345 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b40afa20 con 0x7fc1b4145200 2024-04-25T00:56:14.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1b259c640 1 --2- 172.21.15.112:0/3426278345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b40b23f0 0x7fc1b40add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.251+0000 7fc1abfff640 1 --2- 172.21.15.112:0/3426278345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b4145200 0x7fc1b40b3ce0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009870 tx=0x7fc19c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82edf5d99d99af37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b1d9b640 1 -- 172.21.15.112:0/3426278345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0aa660 con 0x7fc1b4145200 2024-04-25T00:56:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b1d9b640 1 -- 172.21.15.112:0/3426278345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c0af070 con 0x7fc1b4145200 2024-04-25T00:56:14.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b1d9b640 1 -- 172.21.15.112:0/3426278345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0aaa20 con 0x7fc1b4145200 2024-04-25T00:56:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3426278345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b4145200 msgr2=0x7fc1b40b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3426278345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b4145200 0x7fc1b40b3ce0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009870 tx=0x7fc19c01c520 comp rx=0 tx=0).stop 2024-04-25T00:56:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3426278345 shutdown_connections 2024-04-25T00:56:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3426278345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b4145200 0x7fc1b40b3ce0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3426278345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b40b23f0 0x7fc1b40add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3426278345 >> 172.21.15.112:0/3426278345 conn(0x7fc1b40b5980 msgr2=0x7fc1b40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3426278345 shutdown_connections 2024-04-25T00:56:14.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3426278345 wait complete. 2024-04-25T00:56:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 Processor -- start 2024-04-25T00:56:14.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- start start 2024-04-25T00:56:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1b4072760 0x7fc1b413ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b23f0 0x7fc1b41e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.258 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b4145200 0x7fc1b4136d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b4152d70 con 0x7fc1b40b23f0 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc1b40b5360 con 0x7fc1b4072760 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc1b4152ef0 con 0x7fc1b4145200 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b23f0 0x7fc1b41e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b23f0 0x7fc1b41e35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33548/0 (socket says 172.21.15.112:33548) 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 -- 172.21.15.112:0/3728732442 learned_addr learned my addr 172.21.15.112:0/3728732442 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b2d9d640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b4145200 0x7fc1b4136d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b259c640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1b4072760 0x7fc1b413ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 -- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b4145200 msgr2=0x7fc1b4136d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b4145200 0x7fc1b4136d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 -- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1b4072760 msgr2=0x7fc1b413ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1b4072760 0x7fc1b413ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 -- 172.21.15.112:0/3728732442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1b414f8f0 con 0x7fc1b40b23f0 2024-04-25T00:56:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b259c640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1b4072760 0x7fc1b413ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1abfff640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b23f0 0x7fc1b41e35f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0ac040 tx=0x7fc19c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1aaffd640 1 -- 172.21.15.112:0/3728732442 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0b4810 con 0x7fc1b40b23f0 2024-04-25T00:56:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1aaffd640 1 -- 172.21.15.112:0/3728732442 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc19c0b49f0 con 0x7fc1b40b23f0 2024-04-25T00:56:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1aaffd640 1 -- 172.21.15.112:0/3728732442 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0b4de0 con 0x7fc1b40b23f0 2024-04-25T00:56:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1b414e340 con 0x7fc1b40b23f0 2024-04-25T00:56:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.255+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1b414fb60 con 0x7fc1b40b23f0 2024-04-25T00:56:14.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.259+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1b404f580 con 0x7fc1b40b23f0 2024-04-25T00:56:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.259+0000 7fc1aaffd640 1 -- 172.21.15.112:0/3728732442 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc19c0bc020 con 0x7fc1b40b23f0 2024-04-25T00:56:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.259+0000 7fc1aaffd640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc1940678c0 0x7fc194069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.259+0000 7fc1aaffd640 1 -- 172.21.15.112:0/3728732442 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fc19c120260 con 0x7fc1b40b23f0 2024-04-25T00:56:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.259+0000 7fc1b259c640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc1940678c0 0x7fc194069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.259+0000 7fc1b259c640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc1940678c0 0x7fc194069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4008930 tx=0x7fc1a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.263+0000 7fc1aaffd640 1 -- 172.21.15.112:0/3728732442 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc19c016110 con 0x7fc1b40b23f0 2024-04-25T00:56:14.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.463+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fc1b414e7f0 con 0x7fc1b40b23f0 2024-04-25T00:56:14.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.467+0000 7fc1aaffd640 1 -- 172.21.15.112:0/3728732442 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fc19c0f12a0 con 0x7fc1b40b23f0 2024-04-25T00:56:14.470 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc1940678c0 msgr2=0x7fc194069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc1940678c0 0x7fc194069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4008930 tx=0x7fc1a4008040 comp rx=0 tx=0).stop 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b23f0 msgr2=0x7fc1b41e35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b23f0 0x7fc1b41e35f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0ac040 tx=0x7fc19c0192c0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 shutdown_connections 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc1b4145200 0x7fc1b4136d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1b40b23f0 0x7fc1b41e35f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc1940678c0 0x7fc194069d80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 --2- 172.21.15.112:0/3728732442 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc1b4072760 0x7fc1b413ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 >> 172.21.15.112:0/3728732442 conn(0x7fc1b40b5980 msgr2=0x7fc1b4144750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 shutdown_connections 2024-04-25T00:56:14.471 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.471+0000 7fc1b8dbd640 1 -- 172.21.15.112:0/3728732442 wait complete. 2024-04-25T00:56:14.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T00:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b4d97c640 1 Processor -- start 2024-04-25T00:56:14.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b4d97c640 1 -- start start 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b4d97c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3b4814b2e0 0x7f3b480bc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b4d97c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f3b4809aa30 0x7f3b4809aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b4d97c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4809bf00 con 0x7f3b48072310 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b4d97c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f3b480bbe30 con 0x7f3b4814b2e0 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b4d97c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f3b480bb500 con 0x7f3b4809aa30 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b46ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3b4814b2e0 0x7f3b480bc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b477fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3b48072310 0x7f3b48072730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41264/0 (socket says 172.21.15.112:41264) 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b477fe640 1 -- 172.21.15.112:0/1735468822 learned_addr learned my addr 172.21.15.112:0/1735468822 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b467fc640 1 --2- 172.21.15.112:0/1735468822 >> v2:172.21.15.187:3300/0 conn(0x7f3b4809aa30 0x7f3b4809aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b45ffb640 1 -- 172.21.15.112:0/1735468822 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094340624 0 0) 0x7f3b4809bf00 con 0x7f3b48072310 2024-04-25T00:56:14.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b45ffb640 1 -- 172.21.15.112:0/1735468822 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b30003530 con 0x7f3b48072310 2024-04-25T00:56:14.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b46ffd640 1 -- 172.21.15.112:0/1735468822 >> v2:172.21.15.187:3300/0 conn(0x7f3b4809aa30 msgr2=0x7f3b4809aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:14.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b46ffd640 1 --2- 172.21.15.112:0/1735468822 >> v2:172.21.15.187:3300/0 conn(0x7f3b4809aa30 0x7f3b4809aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b46ffd640 1 -- 172.21.15.112:0/1735468822 >> v1:172.21.15.112:6789/0 conn(0x7f3b48072310 legacy=0x7f3b48072730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.607+0000 7f3b46ffd640 1 -- 172.21.15.112:0/1735468822 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4809c1e0 con 0x7f3b4814b2e0 2024-04-25T00:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b46ffd640 1 --2- 172.21.15.112:0/1735468822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4814b2e0 0x7f3b480bc2c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3b480b3ea0 tx=0x7f3b3801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca9f65b8235de41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b45ffb640 1 -- 172.21.15.112:0/1735468822 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380b2070 con 0x7f3b4814b2e0 2024-04-25T00:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b45ffb640 1 -- 172.21.15.112:0/1735468822 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b380ad680 con 0x7f3b4814b2e0 2024-04-25T00:56:14.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b45ffb640 1 -- 172.21.15.112:0/1735468822 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380ada50 con 0x7f3b4814b2e0 2024-04-25T00:56:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- 172.21.15.112:0/1735468822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4814b2e0 msgr2=0x7f3b480bc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/1735468822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4814b2e0 0x7f3b480bc2c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3b480b3ea0 tx=0x7f3b3801c3f0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- 172.21.15.112:0/1735468822 shutdown_connections 2024-04-25T00:56:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/1735468822 >> v2:172.21.15.187:3300/0 conn(0x7f3b4809aa30 0x7f3b4809aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/1735468822 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4814b2e0 0x7f3b480bc2c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- 172.21.15.112:0/1735468822 >> 172.21.15.112:0/1735468822 conn(0x7f3b48067710 msgr2=0x7f3b4809e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- 172.21.15.112:0/1735468822 shutdown_connections 2024-04-25T00:56:14.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- 172.21.15.112:0/1735468822 wait complete. 2024-04-25T00:56:14.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 Processor -- start 2024-04-25T00:56:14.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- start start 2024-04-25T00:56:14.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b48072310 0x7f3b4813e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4809aa30 0x7f3b480b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b4814b2e0 0x7f3b480b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f3b480bc800 con 0x7f3b4809aa30 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4809b400 con 0x7f3b4814b2e0 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b4d97c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f3b480bc980 con 0x7f3b48072310 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4809aa30 0x7f3b480b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4809aa30 0x7f3b480b2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33568/0 (socket says 172.21.15.112:33568) 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 -- 172.21.15.112:0/2303530412 learned_addr learned my addr 172.21.15.112:0/2303530412 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 -- 172.21.15.112:0/2303530412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b48072310 msgr2=0x7f3b4813e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b48072310 0x7f3b4813e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 -- 172.21.15.112:0/2303530412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b4814b2e0 msgr2=0x7f3b480b8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b4814b2e0 0x7f3b480b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.615 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 -- 172.21.15.112:0/2303530412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4813f820 con 0x7f3b4809aa30 2024-04-25T00:56:14.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.611+0000 7f3b467fc640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4809aa30 0x7f3b480b2ff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00d830 tx=0x7f3b3c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b2ffff640 1 -- 172.21.15.112:0/2303530412 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00b6c0 con 0x7f3b4809aa30 2024-04-25T00:56:14.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b2ffff640 1 -- 172.21.15.112:0/2303530412 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3c019070 con 0x7f3b4809aa30 2024-04-25T00:56:14.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4813fa90 con 0x7f3b4809aa30 2024-04-25T00:56:14.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b2ffff640 1 -- 172.21.15.112:0/2303530412 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c00ba60 con 0x7f3b4809aa30 2024-04-25T00:56:14.616 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b48140430 con 0x7f3b4809aa30 2024-04-25T00:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4804f580 con 0x7f3b4809aa30 2024-04-25T00:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b2ffff640 1 -- 172.21.15.112:0/2303530412 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3b3c01e460 con 0x7f3b4809aa30 2024-04-25T00:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b2ffff640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b340678c0 0x7f3b34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b2ffff640 1 -- 172.21.15.112:0/2303530412 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f3b3c085fb0 con 0x7f3b4809aa30 2024-04-25T00:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b46ffd640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b340678c0 0x7f3b34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.615+0000 7f3b46ffd640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b340678c0 0x7f3b34069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3b380af000 tx=0x7f3b38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.619+0000 7f3b2ffff640 1 -- 172.21.15.112:0/2303530412 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3b3c056ed0 con 0x7f3b4809aa30 2024-04-25T00:56:14.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.835+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f3b480b7060 con 0x7f3b4809aa30 2024-04-25T00:56:14.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.839+0000 7f3b2ffff640 1 -- 172.21.15.112:0/2303530412 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f3b3c0567d0 con 0x7f3b4809aa30 2024-04-25T00:56:14.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b340678c0 msgr2=0x7f3b34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b340678c0 0x7f3b34069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3b380af000 tx=0x7f3b38008040 comp rx=0 tx=0).stop 2024-04-25T00:56:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4809aa30 msgr2=0x7f3b480b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4809aa30 0x7f3b480b2ff0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c00d830 tx=0x7f3b3c00dcf0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 shutdown_connections 2024-04-25T00:56:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f3b340678c0 0x7f3b34069d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f3b4814b2e0 0x7f3b480b8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3b4809aa30 0x7f3b480b2ff0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.844 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 --2- 172.21.15.112:0/2303530412 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f3b48072310 0x7f3b4813e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 >> 172.21.15.112:0/2303530412 conn(0x7f3b48067710 msgr2=0x7f3b48147bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:14.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 shutdown_connections 2024-04-25T00:56:14.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.843+0000 7f3b4d97c640 1 -- 172.21.15.112:0/2303530412 wait complete. 2024-04-25T00:56:14.860 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T00:56:14.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd299843640 1 Processor -- start 2024-04-25T00:56:14.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd299843640 1 -- start start 2024-04-25T00:56:14.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd299843640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fd2940b23f0 0x7fd2940add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd299843640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd294145200 0x7fd2940b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd299843640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2940aeec0 con 0x7fd294072760 2024-04-25T00:56:14.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd299843640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd2940af780 con 0x7fd294145200 2024-04-25T00:56:14.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd299843640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fd2940b27d0 con 0x7fd2940b23f0 2024-04-25T00:56:14.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd2927fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd294145200 0x7fd2940b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd2927fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd294145200 0x7fd2940b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33580/0 (socket says 172.21.15.112:33580) 2024-04-25T00:56:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd2927fc640 1 -- 172.21.15.112:0/1019033886 learned_addr learned my addr 172.21.15.112:0/1019033886 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd292ffd640 1 --2- 172.21.15.112:0/1019033886 >> v2:172.21.15.187:3300/0 conn(0x7fd2940b23f0 0x7fd2940add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd2927fc640 1 -- 172.21.15.112:0/1019033886 >> v2:172.21.15.187:3300/0 conn(0x7fd2940b23f0 msgr2=0x7fd2940add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd2927fc640 1 --2- 172.21.15.112:0/1019033886 >> v2:172.21.15.187:3300/0 conn(0x7fd2940b23f0 0x7fd2940add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd2927fc640 1 -- 172.21.15.112:0/1019033886 >> v1:172.21.15.187:6789/0 conn(0x7fd294072760 legacy=0x7fd29406c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:14.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.979+0000 7fd2927fc640 1 -- 172.21.15.112:0/1019033886 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2940afa20 con 0x7fd294145200 2024-04-25T00:56:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 --2- 172.21.15.112:0/1019033886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd294145200 0x7fd2940b3ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd29406d9f0 tx=0x7fd28401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb98df56f47ce1c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd291ffb640 1 -- 172.21.15.112:0/1019033886 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2840ab5f0 con 0x7fd294145200 2024-04-25T00:56:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd291ffb640 1 -- 172.21.15.112:0/1019033886 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2840b0070 con 0x7fd294145200 2024-04-25T00:56:14.984 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd291ffb640 1 -- 172.21.15.112:0/1019033886 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2840ab9b0 con 0x7fd294145200 2024-04-25T00:56:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- 172.21.15.112:0/1019033886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd294145200 msgr2=0x7fd2940b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 --2- 172.21.15.112:0/1019033886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd294145200 0x7fd2940b3ce0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd29406d9f0 tx=0x7fd28401c4e0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- 172.21.15.112:0/1019033886 shutdown_connections 2024-04-25T00:56:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 --2- 172.21.15.112:0/1019033886 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd294145200 0x7fd2940b3ce0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 --2- 172.21.15.112:0/1019033886 >> v2:172.21.15.187:3300/0 conn(0x7fd2940b23f0 0x7fd2940add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- 172.21.15.112:0/1019033886 >> 172.21.15.112:0/1019033886 conn(0x7fd2940b5980 msgr2=0x7fd2940b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- 172.21.15.112:0/1019033886 shutdown_connections 2024-04-25T00:56:14.985 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- 172.21.15.112:0/1019033886 wait complete. 2024-04-25T00:56:14.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 Processor -- start 2024-04-25T00:56:14.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- start start 2024-04-25T00:56:14.989 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd294072760 0x7fd29413e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940b23f0 0x7fd29413a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd294145200 0x7fd294136c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd294072f70 con 0x7fd2940b23f0 2024-04-25T00:56:14.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd2940b5360 con 0x7fd294145200 2024-04-25T00:56:14.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd299843640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd294152d70 con 0x7fd294072760 2024-04-25T00:56:14.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940b23f0 0x7fd29413a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940b23f0 0x7fd29413a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33586/0 (socket says 172.21.15.112:33586) 2024-04-25T00:56:14.990 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 -- 172.21.15.112:0/6617345 learned_addr learned my addr 172.21.15.112:0/6617345 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd292ffd640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd294072760 0x7fd29413e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2937fe640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd294145200 0x7fd294136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 -- 172.21.15.112:0/6617345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd294072760 msgr2=0x7fd29413e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd294072760 0x7fd29413e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 -- 172.21.15.112:0/6617345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd294145200 msgr2=0x7fd294136c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd294145200 0x7fd294136c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.983+0000 7fd2927fc640 1 -- 172.21.15.112:0/6617345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29413a1f0 con 0x7fd2940b23f0 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd2927fc640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940b23f0 0x7fd29413a8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd28401a110 tx=0x7fd284019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.991 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd27bfff640 1 -- 172.21.15.112:0/6617345 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2840bf610 con 0x7fd2940b23f0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd27bfff640 1 -- 172.21.15.112:0/6617345 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2840bf7b0 con 0x7fd2940b23f0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd294071b20 con 0x7fd2940b23f0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd27bfff640 1 -- 172.21.15.112:0/6617345 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2840bfb80 con 0x7fd2940b23f0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd29413bb00 con 0x7fd2940b23f0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd29404f580 con 0x7fd2940b23f0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd27bfff640 1 -- 172.21.15.112:0/6617345 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd2840bd020 con 0x7fd2940b23f0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd27bfff640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd2800678c0 0x7fd280069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd27bfff640 1 -- 172.21.15.112:0/6617345 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fd28411fd20 con 0x7fd2940b23f0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd292ffd640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd2800678c0 0x7fd280069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:14.992 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.987+0000 7fd292ffd640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd2800678c0 0x7fd280069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd28800a970 tx=0x7fd288008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:14.993 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:14.991+0000 7fd27bfff640 1 -- 172.21.15.112:0/6617345 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd2840c4060 con 0x7fd2940b23f0 2024-04-25T00:56:15.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.207+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd29413bd30 con 0x7fd2940b23f0 2024-04-25T00:56:15.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.211+0000 7fd27bfff640 1 -- 172.21.15.112:0/6617345 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fd2840160d0 con 0x7fd2940b23f0 2024-04-25T00:56:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.211+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd2800678c0 msgr2=0x7fd280069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.211+0000 7fd299843640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd2800678c0 0x7fd280069d80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd28800a970 tx=0x7fd288008040 comp rx=0 tx=0).stop 2024-04-25T00:56:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.211+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940b23f0 msgr2=0x7fd29413a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.211+0000 7fd299843640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940b23f0 0x7fd29413a8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd28401a110 tx=0x7fd284019280 comp rx=0 tx=0).stop 2024-04-25T00:56:15.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.215+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 shutdown_connections 2024-04-25T00:56:15.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.215+0000 7fd299843640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd2800678c0 0x7fd280069d80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.215+0000 7fd299843640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd294145200 0x7fd294136c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.215+0000 7fd299843640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2940b23f0 0x7fd29413a8f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.215+0000 7fd299843640 1 --2- 172.21.15.112:0/6617345 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd294072760 0x7fd29413e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.215+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 >> 172.21.15.112:0/6617345 conn(0x7fd2940b5980 msgr2=0x7fd294144750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:15.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.215+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 shutdown_connections 2024-04-25T00:56:15.215 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.215+0000 7fd299843640 1 -- 172.21.15.112:0/6617345 wait complete. 2024-04-25T00:56:15.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T00:56:15.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f4511432640 1 Processor -- start 2024-04-25T00:56:15.344 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f4511432640 1 -- start start 2024-04-25T00:56:15.345 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f4511432640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f450c0add70 0x7f450c145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f4511432640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f450c0b3ce0 0x7f450c14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f4511432640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f450c0aeec0 con 0x7f450c0b23f0 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f4511432640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f450c0af780 con 0x7f450c0add70 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f4511432640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f450c0b27d0 con 0x7f450c0b3ce0 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450a7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f450c0add70 0x7f450c145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450a7fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f450c0add70 0x7f450c145200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33592/0 (socket says 172.21.15.112:33592) 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450a7fc640 1 -- 172.21.15.112:0/1885630297 learned_addr learned my addr 172.21.15.112:0/1885630297 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450affd640 1 --1- 172.21.15.112:0/1885630297 >> v1:172.21.15.112:6789/0 conn(0x7f450c0b23f0 0x7f450c072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41274/0 (socket says 172.21.15.112:41274) 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450b7fe640 1 --2- 172.21.15.112:0/1885630297 >> v2:172.21.15.187:3300/0 conn(0x7f450c0b3ce0 0x7f450c14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450a7fc640 1 -- 172.21.15.112:0/1885630297 >> v2:172.21.15.187:3300/0 conn(0x7f450c0b3ce0 msgr2=0x7f450c14fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450a7fc640 1 --2- 172.21.15.112:0/1885630297 >> v2:172.21.15.187:3300/0 conn(0x7f450c0b3ce0 0x7f450c14fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450a7fc640 1 -- 172.21.15.112:0/1885630297 >> v1:172.21.15.112:6789/0 conn(0x7f450c0b23f0 legacy=0x7f450c072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.346 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450a7fc640 1 -- 172.21.15.112:0/1885630297 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c0afa20 con 0x7f450c0add70 2024-04-25T00:56:15.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f450b7fe640 1 --2- 172.21.15.112:0/1885630297 >> v2:172.21.15.187:3300/0 conn(0x7f450c0b3ce0 0x7f450c14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:15.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.343+0000 7f4509ffb640 1 -- 172.21.15.112:0/1885630297 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599809794 0 0) 0x7f450c0aeec0 con 0x7f450c0b23f0 2024-04-25T00:56:15.347 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f450a7fc640 1 --2- 172.21.15.112:0/1885630297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0add70 0x7f450c145200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f450c0b38b0 tx=0x7f44f401cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944a2cc1525a5816 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4509ffb640 1 -- 172.21.15.112:0/1885630297 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40a9400 con 0x7f450c0add70 2024-04-25T00:56:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4509ffb640 1 -- 172.21.15.112:0/1885630297 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f40a95a0 con 0x7f450c0add70 2024-04-25T00:56:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4509ffb640 1 -- 172.21.15.112:0/1885630297 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40a9970 con 0x7f450c0add70 2024-04-25T00:56:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- 172.21.15.112:0/1885630297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0add70 msgr2=0x7f450c145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.348 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 --2- 172.21.15.112:0/1885630297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0add70 0x7f450c145200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f450c0b38b0 tx=0x7f44f401cf80 comp rx=0 tx=0).stop 2024-04-25T00:56:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- 172.21.15.112:0/1885630297 shutdown_connections 2024-04-25T00:56:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 --2- 172.21.15.112:0/1885630297 >> v2:172.21.15.187:3300/0 conn(0x7f450c0b3ce0 0x7f450c14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 --2- 172.21.15.112:0/1885630297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0add70 0x7f450c145200 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- 172.21.15.112:0/1885630297 >> 172.21.15.112:0/1885630297 conn(0x7f450c0b5980 msgr2=0x7f450c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- 172.21.15.112:0/1885630297 shutdown_connections 2024-04-25T00:56:15.349 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- 172.21.15.112:0/1885630297 wait complete. 2024-04-25T00:56:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 Processor -- start 2024-04-25T00:56:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- start start 2024-04-25T00:56:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450c0add70 0x7f450c0713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f450c0b23f0 0x7f450c1e0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0b3ce0 0x7f450c1e0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f450c0b52d0 con 0x7f450c0b3ce0 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f450c0b5150 con 0x7f450c0add70 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.347+0000 7f4511432640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f450c1528a0 con 0x7f450c0b23f0 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0b3ce0 0x7f450c1e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0b3ce0 0x7f450c1e0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33604/0 (socket says 172.21.15.112:33604) 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 -- 172.21.15.112:0/3948331722 learned_addr learned my addr 172.21.15.112:0/3948331722 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450affd640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450c0add70 0x7f450c0713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 -- 172.21.15.112:0/3948331722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f450c0b23f0 msgr2=0x7f450c1e0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f450c0b23f0 0x7f450c1e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.352 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 -- 172.21.15.112:0/3948331722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450c0add70 msgr2=0x7f450c0713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450c0add70 0x7f450c0713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 -- 172.21.15.112:0/3948331722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c0b7600 con 0x7f450c0b3ce0 2024-04-25T00:56:15.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450b7fe640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0b3ce0 0x7f450c1e0560 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4504007d70 tx=0x7f450400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:15.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f45037fe640 1 -- 172.21.15.112:0/3948331722 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450401a070 con 0x7f450c0b3ce0 2024-04-25T00:56:15.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f45037fe640 1 -- 172.21.15.112:0/3948331722 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45040039a0 con 0x7f450c0b3ce0 2024-04-25T00:56:15.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450c070660 con 0x7f450c0b3ce0 2024-04-25T00:56:15.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f45037fe640 1 -- 172.21.15.112:0/3948331722 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4504003d90 con 0x7f450c0b3ce0 2024-04-25T00:56:15.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450c0b7320 con 0x7f450c0b3ce0 2024-04-25T00:56:15.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450c04f580 con 0x7f450c0b3ce0 2024-04-25T00:56:15.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f45037fe640 1 -- 172.21.15.112:0/3948331722 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f450401d070 con 0x7f450c0b3ce0 2024-04-25T00:56:15.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f45037fe640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f44ec0678c0 0x7f44ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f45037fe640 1 -- 172.21.15.112:0/3948331722 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f4504085b00 con 0x7f450c0b3ce0 2024-04-25T00:56:15.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450affd640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f44ec0678c0 0x7f44ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.351+0000 7f450affd640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f44ec0678c0 0x7f44ec069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000f70 tx=0x7f44fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:15.357 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.355+0000 7f45037fe640 1 -- 172.21.15.112:0/3948331722 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4504010070 con 0x7f450c0b3ce0 2024-04-25T00:56:15.575 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.571+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f450c0b16d0 con 0x7f450c0b3ce0 2024-04-25T00:56:15.576 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.575+0000 7f45037fe640 1 -- 172.21.15.112:0/3948331722 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f4504056990 con 0x7f450c0b3ce0 2024-04-25T00:56:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f44ec0678c0 msgr2=0x7f44ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f44ec0678c0 0x7f44ec069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000f70 tx=0x7f44fc009040 comp rx=0 tx=0).stop 2024-04-25T00:56:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0b3ce0 msgr2=0x7f450c1e0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0b3ce0 0x7f450c1e0560 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4504007d70 tx=0x7f450400d4a0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 shutdown_connections 2024-04-25T00:56:15.580 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f450c0b3ce0 0x7f450c1e0560 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f450c0b23f0 0x7f450c1e0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f44ec0678c0 0x7f44ec069d80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 --2- 172.21.15.112:0/3948331722 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f450c0add70 0x7f450c0713d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 >> 172.21.15.112:0/3948331722 conn(0x7f450c0b5980 msgr2=0x7f450c06c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 shutdown_connections 2024-04-25T00:56:15.581 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.579+0000 7f4511432640 1 -- 172.21.15.112:0/3948331722 wait complete. 2024-04-25T00:56:15.596 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:15.705 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.703+0000 7fe55918e640 1 Processor -- start 2024-04-25T00:56:15.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.703+0000 7fe55918e640 1 -- start start 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.703+0000 7fe55918e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe5540b23f0 0x7fe5540add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.703+0000 7fe55918e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5540aeec0 con 0x7fe554145200 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55918e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5540af780 con 0x7fe5540b3ce0 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55918e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe5540b27d0 con 0x7fe5540b23f0 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55259c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fe554145200 0x7fe554072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41278/0 (socket says 172.21.15.112:41278) 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55259c640 1 -- 172.21.15.112:0/2178754214 learned_addr learned my addr 172.21.15.112:0/2178754214 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe552d9d640 1 --2- 172.21.15.112:0/2178754214 >> v2:172.21.15.187:3300/0 conn(0x7fe5540b23f0 0x7fe5540add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1250184380 0 0) 0x7fe5540aeec0 con 0x7fe554145200 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe53c003530 con 0x7fe554145200 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2499229745 0 0) 0x7fe53c003530 con 0x7fe554145200 2024-04-25T00:56:15.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 >> v2:172.21.15.187:3300/0 conn(0x7fe5540b23f0 msgr2=0x7fe5540add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 --2- 172.21.15.112:0/2178754214 >> v2:172.21.15.187:3300/0 conn(0x7fe5540b23f0 0x7fe5540add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 >> v1:172.21.15.187:6789/0 conn(0x7fe5540b3ce0 legacy=0x7fe55414fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540afa20 con 0x7fe554145200 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7fe544001680 con 0x7fe554145200 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 >> v1:172.21.15.112:6789/0 conn(0x7fe554145200 legacy=0x7fe554072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53c0039c0 0x7fe53c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe53c004810 0x7fe53c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53c0051d0 0x7fe53c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540b27d0 con 0x7fe53c0039c0 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c0104b0 con 0x7fe53c0051d0 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c0106b0 con 0x7fe53c004810 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe552d9d640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53c0039c0 0x7fe53c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe552d9d640 1 -- 172.21.15.112:0/2178754214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe53c004810 msgr2=0x7fe53c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:15.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe552d9d640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe53c004810 0x7fe53c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe552d9d640 1 -- 172.21.15.112:0/2178754214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53c0051d0 msgr2=0x7fe53c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:15.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe552d9d640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe53c0051d0 0x7fe53c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe552d9d640 1 -- 172.21.15.112:0/2178754214 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c003530 con 0x7fe53c0039c0 2024-04-25T00:56:15.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55359e640 1 -- 172.21.15.112:0/2178754214 reap_dead start 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe552d9d640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53c0039c0 0x7fe53c003e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe548009ae0 tx=0x7fe54801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa113fa89c93b47f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5480b2070 con 0x7fe53c0039c0 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5480a5570 con 0x7fe53c0039c0 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe551d9b640 1 -- 172.21.15.112:0/2178754214 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5480096b0 con 0x7fe53c0039c0 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55918e640 1 -- 172.21.15.112:0/2178754214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53c0039c0 msgr2=0x7fe53c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55918e640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53c0039c0 0x7fe53c003e00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe548009ae0 tx=0x7fe54801ca90 comp rx=0 tx=0).stop 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55918e640 1 -- 172.21.15.112:0/2178754214 shutdown_connections 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55918e640 1 --2- 172.21.15.112:0/2178754214 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53c0039c0 0x7fe53c003e00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.707+0000 7fe55918e640 1 -- 172.21.15.112:0/2178754214 >> 172.21.15.112:0/2178754214 conn(0x7fe5540b5980 msgr2=0x7fe5540b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- 172.21.15.112:0/2178754214 shutdown_connections 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- 172.21.15.112:0/2178754214 wait complete. 2024-04-25T00:56:15.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 Processor -- start 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- start start 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe53c0039c0 0x7fe5541477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5541d1d20 0x7fe5541485f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe554139d50 0x7fe55413a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540b1f90 con 0x7fe554139d50 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe5540b5360 con 0x7fe5541d1d20 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe55406c2c0 con 0x7fe53c0039c0 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe554139d50 0x7fe55413a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe554139d50 0x7fe55413a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33624/0 (socket says 172.21.15.112:33624) 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 -- 172.21.15.112:0/2737573215 learned_addr learned my addr 172.21.15.112:0/2737573215 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe552d9d640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe53c0039c0 0x7fe5541477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55259c640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5541d1d20 0x7fe5541485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 -- 172.21.15.112:0/2737573215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe53c0039c0 msgr2=0x7fe5541477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe53c0039c0 0x7fe5541477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 -- 172.21.15.112:0/2737573215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5541d1d20 msgr2=0x7fe5541485f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5541d1d20 0x7fe5541485f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 -- 172.21.15.112:0/2737573215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe554147e10 con 0x7fe554139d50 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55359e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe554139d50 0x7fe55413a1d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe55406f140 tx=0x7fe54c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe53b7fe640 1 -- 172.21.15.112:0/2737573215 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c007110 con 0x7fe554139d50 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe53b7fe640 1 -- 172.21.15.112:0/2737573215 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54c0072d0 con 0x7fe554139d50 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe554146380 con 0x7fe554139d50 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe53b7fe640 1 -- 172.21.15.112:0/2737573215 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0076c0 con 0x7fe554139d50 2024-04-25T00:56:15.715 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55413a710 con 0x7fe554139d50 2024-04-25T00:56:15.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55404f580 con 0x7fe554139d50 2024-04-25T00:56:15.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.711+0000 7fe53b7fe640 1 -- 172.21.15.112:0/2737573215 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe54c011070 con 0x7fe554139d50 2024-04-25T00:56:15.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.715+0000 7fe53b7fe640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5400678c0 0x7fe540069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:15.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.715+0000 7fe53b7fe640 1 -- 172.21.15.112:0/2737573215 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fe54c085080 con 0x7fe554139d50 2024-04-25T00:56:15.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.715+0000 7fe552d9d640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5400678c0 0x7fe540069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:15.716 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.715+0000 7fe552d9d640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5400678c0 0x7fe540069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe548009ae0 tx=0x7fe548008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:15.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.719+0000 7fe53b7fe640 1 -- 172.21.15.112:0/2737573215 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe54c019060 con 0x7fe554139d50 2024-04-25T00:56:15.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.871+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5540b7c00 con 0x7fe554139d50 2024-04-25T00:56:15.874 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe53b7fe640 1 -- 172.21.15.112:0/2737573215 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+12939 (secure 0 0 0) 0x7fe54c055fa0 con 0x7fe554139d50 2024-04-25T00:56:15.874 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:15.875 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":25,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:13.023485+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:15.876 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5400678c0 msgr2=0x7fe540069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5400678c0 0x7fe540069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe548009ae0 tx=0x7fe548008040 comp rx=0 tx=0).stop 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe554139d50 msgr2=0x7fe55413a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe554139d50 0x7fe55413a1d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe55406f140 tx=0x7fe54c00c8e0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 shutdown_connections 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe554139d50 0x7fe55413a1d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5400678c0 0x7fe540069d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe5541d1d20 0x7fe5541485f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 --2- 172.21.15.112:0/2737573215 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe53c0039c0 0x7fe5541477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 >> 172.21.15.112:0/2737573215 conn(0x7fe5540b5980 msgr2=0x7fe5540b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 shutdown_connections 2024-04-25T00:56:15.877 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:15.875+0000 7fe55918e640 1 -- 172.21.15.112:0/2737573215 wait complete. 2024-04-25T00:56:15.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:56:15.892 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:56:16.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cd4e97640 1 Processor -- start 2024-04-25T00:56:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cd4e97640 1 -- start start 2024-04-25T00:56:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cd4e97640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00affb0 0x7f6cd009da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cd4e97640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6cd00b0390 0x7f6cd014fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cd4e97640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd00b1390 con 0x7f6cd009dfa0 2024-04-25T00:56:16.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cd4e97640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd00b1d70 con 0x7f6cd00b0390 2024-04-25T00:56:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cd4e97640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd00b1600 con 0x7f6cd00affb0 2024-04-25T00:56:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6ccdd9b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6cd009dfa0 0x7f6cd0148800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41294/0 (socket says 172.21.15.112:41294) 2024-04-25T00:56:16.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6ccdd9b640 1 -- 172.21.15.112:0/211367386 learned_addr learned my addr 172.21.15.112:0/211367386 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 268836998 0 0) 0x7f6cd00b1390 con 0x7f6cd009dfa0 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cce59c640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00affb0 0x7f6cd009da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6cced9d640 1 --2- 172.21.15.112:0/211367386 >> v2:172.21.15.187:3300/0 conn(0x7f6cd00b0390 0x7f6cd014fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cb8003530 con 0x7f6cd009dfa0 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2074196391 0 0) 0x7f6cb8003530 con 0x7f6cd009dfa0 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00affb0 msgr2=0x7f6cd009da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.003+0000 7f6ccd59a640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00affb0 0x7f6cd009da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 >> v2:172.21.15.187:3300/0 conn(0x7f6cd00b0390 msgr2=0x7f6cd014fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 --2- 172.21.15.112:0/211367386 >> v2:172.21.15.187:3300/0 conn(0x7f6cd00b0390 0x7f6cd014fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd00b1fa0 con 0x7f6cd009dfa0 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f6cc4001680 con 0x7f6cd009dfa0 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 >> v1:172.21.15.112:6789/0 conn(0x7f6cd009dfa0 legacy=0x7f6cd0148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00affb0 0x7f6cb80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8004020 0x7f6cb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb8004d50 0x7f6cb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd00b1600 con 0x7f6cb8004020 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb800bb60 con 0x7f6cb8004d50 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cb800bd30 con 0x7f6cd00affb0 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8004020 0x7f6cb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cce59c640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00affb0 0x7f6cb80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cced9d640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb8004d50 0x7f6cb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccdd9b640 1 -- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00affb0 msgr2=0x7f6cb80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cd00affb0 0x7f6cb80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccdd9b640 1 -- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb8004d50 msgr2=0x7f6cb80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cb8004d50 0x7f6cb80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.009 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccdd9b640 1 -- 172.21.15.112:0/211367386 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb8003530 con 0x7f6cb8004020 2024-04-25T00:56:16.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8004020 0x7f6cb8004810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c1f0 tx=0x7f6cc401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a59673a7dab7526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.012 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4007d00 con 0x7f6cb8004020 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc40a9020 con 0x7f6cb8004020 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6ccd59a640 1 -- 172.21.15.112:0/211367386 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4005ae0 con 0x7f6cb8004020 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cd4e97640 1 -- 172.21.15.112:0/211367386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8004020 msgr2=0x7f6cb8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cd4e97640 1 --2- 172.21.15.112:0/211367386 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cb8004020 0x7f6cb8004810 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c1f0 tx=0x7f6cc401cc60 comp rx=0 tx=0).stop 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cced9d640 1 -- 172.21.15.112:0/211367386 reap_dead start 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cd4e97640 1 -- 172.21.15.112:0/211367386 shutdown_connections 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cd4e97640 1 -- 172.21.15.112:0/211367386 >> 172.21.15.112:0/211367386 conn(0x7f6cd00b8c80 msgr2=0x7f6cd00b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cd4e97640 1 -- 172.21.15.112:0/211367386 shutdown_connections 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.007+0000 7f6cd4e97640 1 -- 172.21.15.112:0/211367386 wait complete. 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 Processor -- start 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 -- start start 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb8004020 0x7f6cd009b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.013 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd0142cc0 0x7f6cd00704b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00709f0 0x7f6cd006f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd009dbe0 con 0x7f6cd0142cc0 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd009da60 con 0x7f6cd00709f0 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd009dd60 con 0x7f6cb8004020 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd0142cc0 0x7f6cd00704b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd0142cc0 0x7f6cd00704b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33644/0 (socket says 172.21.15.112:33644) 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 -- 172.21.15.112:0/2719616294 learned_addr learned my addr 172.21.15.112:0/2719616294 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cced9d640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00709f0 0x7f6cd006f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 -- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb8004020 msgr2=0x7f6cd009b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cce59c640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb8004020 0x7f6cd009b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb8004020 0x7f6cd009b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 -- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00709f0 msgr2=0x7f6cd006f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.014 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00709f0 0x7f6cd006f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 -- 172.21.15.112:0/2719616294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd009ae10 con 0x7f6cd0142cc0 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cced9d640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00709f0 0x7f6cd006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cce59c640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb8004020 0x7f6cd009b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6ccdd9b640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd0142cc0 0x7f6cd00704b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c190 tx=0x7f6cb8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2719616294 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40a5c60 con 0x7f6cd0142cc0 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2719616294 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc40a5e00 con 0x7f6cd0142cc0 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd009c0d0 con 0x7f6cd0142cc0 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2719616294 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4004a10 con 0x7f6cd0142cc0 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd009bc30 con 0x7f6cd0142cc0 2024-04-25T00:56:16.015 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd00b1040 con 0x7f6cd0142cc0 2024-04-25T00:56:16.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2719616294 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6cc4004bb0 con 0x7f6cd0142cc0 2024-04-25T00:56:16.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cbeffd640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6cb00678c0 0x7f6cb0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2719616294 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f6cc411ecd0 con 0x7f6cd0142cc0 2024-04-25T00:56:16.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.011+0000 7f6cce59c640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6cb00678c0 0x7f6cb0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.016 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.015+0000 7f6cce59c640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6cb00678c0 0x7f6cb0069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8003fd0 tx=0x7f6cc800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.017 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.015+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2719616294 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6cc4016970 con 0x7f6cd0142cc0 2024-04-25T00:56:16.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6cd004f5f0 con 0x7f6cd0142cc0 2024-04-25T00:56:16.168 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cbeffd640 1 -- 172.21.15.112:0/2719616294 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f6cc40efbf0 con 0x7f6cd0142cc0 2024-04-25T00:56:16.168 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:56:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6cb00678c0 msgr2=0x7f6cb0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6cb00678c0 0x7f6cb0069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8003fd0 tx=0x7f6cc800b040 comp rx=0 tx=0).stop 2024-04-25T00:56:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd0142cc0 msgr2=0x7f6cd00704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd0142cc0 0x7f6cd00704b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c190 tx=0x7f6cb8005840 comp rx=0 tx=0).stop 2024-04-25T00:56:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 shutdown_connections 2024-04-25T00:56:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6cb00678c0 0x7f6cb0069d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6cd00709f0 0x7f6cd006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd0142cc0 0x7f6cd00704b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 --2- 172.21.15.112:0/2719616294 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6cb8004020 0x7f6cd009b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 >> 172.21.15.112:0/2719616294 conn(0x7f6cd00b8c80 msgr2=0x7f6cd006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:16.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 shutdown_connections 2024-04-25T00:56:16.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.167+0000 7f6cd4e97640 1 -- 172.21.15.112:0/2719616294 wait complete. 2024-04-25T00:56:16.185 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:56:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.287+0000 7fafcfc40640 1 Processor -- start 2024-04-25T00:56:16.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.287+0000 7fafcfc40640 1 -- start start 2024-04-25T00:56:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.287+0000 7fafcfc40640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc80add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.287+0000 7fafcfc40640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafc80b3ce0 0x7fafc814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.287+0000 7fafcfc40640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc80aeec0 con 0x7fafc8145200 2024-04-25T00:56:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.287+0000 7fafcfc40640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fafc80af780 con 0x7fafc80b3ce0 2024-04-25T00:56:16.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.287+0000 7fafcfc40640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fafc80b27d0 con 0x7fafc80b23f0 2024-04-25T00:56:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcd9dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc80add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafce1dd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafc80b3ce0 0x7fafc814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcd9dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc80add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:52180/0 (socket says 172.21.15.112:52180) 2024-04-25T00:56:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafce1dd640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fafc80b3ce0 0x7fafc814fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33512/0 (socket says 172.21.15.112:33512) 2024-04-25T00:56:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcd9dc640 1 -- 172.21.15.112:0/1917259161 learned_addr learned my addr 172.21.15.112:0/1917259161 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:16.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcd1db640 1 --1- 172.21.15.112:0/1917259161 >> v1:172.21.15.187:6789/0 conn(0x7fafc8145200 0x7fafc8072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:57150/0 (socket says 172.21.15.112:57150) 2024-04-25T00:56:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafce1dd640 1 -- 172.21.15.112:0/1917259161 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 msgr2=0x7fafc80add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafce1dd640 1 --2- 172.21.15.112:0/1917259161 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc80add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafce1dd640 1 -- 172.21.15.112:0/1917259161 >> v1:172.21.15.187:6789/0 conn(0x7fafc8145200 legacy=0x7fafc8072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafce1dd640 1 -- 172.21.15.112:0/1917259161 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc80b4150 con 0x7fafc80b3ce0 2024-04-25T00:56:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafce1dd640 1 --2- 172.21.15.112:0/1917259161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80b3ce0 0x7fafc814fb40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fafc40097c0 tx=0x7fafc401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28a27d43e512aea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcc9da640 1 -- 172.21.15.112:0/1917259161 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc40b1070 con 0x7fafc80b3ce0 2024-04-25T00:56:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcc9da640 1 -- 172.21.15.112:0/1917259161 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc40ac410 con 0x7fafc80b3ce0 2024-04-25T00:56:16.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcc9da640 1 -- 172.21.15.112:0/1917259161 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc40ac7e0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcfc40640 1 -- 172.21.15.112:0/1917259161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80b3ce0 msgr2=0x7fafc814fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1917259161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80b3ce0 0x7fafc814fb40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fafc40097c0 tx=0x7fafc401c4a0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcfc40640 1 -- 172.21.15.112:0/1917259161 shutdown_connections 2024-04-25T00:56:16.294 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1917259161 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc80b3ce0 0x7fafc814fb40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1917259161 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc80add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcfc40640 1 -- 172.21.15.112:0/1917259161 >> 172.21.15.112:0/1917259161 conn(0x7fafc80b5980 msgr2=0x7fafc80b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.291+0000 7fafcfc40640 1 -- 172.21.15.112:0/1917259161 shutdown_connections 2024-04-25T00:56:16.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 -- 172.21.15.112:0/1917259161 wait complete. 2024-04-25T00:56:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 Processor -- start 2024-04-25T00:56:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 -- start start 2024-04-25T00:56:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc813d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc80b3ce0 0x7fafc8148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc8145200 0x7fafc813f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80b52d0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80b5150 con 0x7fafc8145200 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fafc81528a0 con 0x7fafc80b23f0 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc80b3ce0 0x7fafc8148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc80b3ce0 0x7fafc8148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33650/0 (socket says 172.21.15.112:33650) 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 -- 172.21.15.112:0/1695538904 learned_addr learned my addr 172.21.15.112:0/1695538904 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd9dc640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc813d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafce1dd640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc8145200 0x7fafc813f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 -- 172.21.15.112:0/1695538904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 msgr2=0x7fafc813d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc813d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 -- 172.21.15.112:0/1695538904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc8145200 msgr2=0x7fafc813f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc8145200 0x7fafc813f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 -- 172.21.15.112:0/1695538904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc81394d0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcd1db640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc80b3ce0 0x7fafc8148f90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fafb800c3a0 tx=0x7fafb800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafbe7fc640 1 -- 172.21.15.112:0/1695538904 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80074e0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafbe7fc640 1 -- 172.21.15.112:0/1695538904 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb8007680 con 0x7fafc80b3ce0 2024-04-25T00:56:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc81391f0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafbe7fc640 1 -- 172.21.15.112:0/1695538904 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8004a30 con 0x7fafc80b3ce0 2024-04-25T00:56:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc81399d0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.295+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc804f580 con 0x7fafc80b3ce0 2024-04-25T00:56:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.299+0000 7fafbe7fc640 1 -- 172.21.15.112:0/1695538904 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fafb801a020 con 0x7fafc80b3ce0 2024-04-25T00:56:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.299+0000 7fafbe7fc640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fafac0678c0 0x7fafac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.299+0000 7fafbe7fc640 1 -- 172.21.15.112:0/1695538904 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7fafb8084f10 con 0x7fafc80b3ce0 2024-04-25T00:56:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.299+0000 7fafcd9dc640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fafac0678c0 0x7fafac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.299+0000 7fafcd9dc640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fafac0678c0 0x7fafac069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fafc00113e0 tx=0x7fafc000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.303+0000 7fafbe7fc640 1 -- 172.21.15.112:0/1695538904 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fafb80557e0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fafc806b9c0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafbe7fc640 1 -- 172.21.15.112:0/1695538904 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v25) ==== 84+0+65 (secure 0 0 0) 0x7fafb8059ae0 con 0x7fafc80b3ce0 2024-04-25T00:56:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:16.452 INFO:teuthology.orchestra.run.smithi112.stdout:[ 2024-04-25T00:56:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: { 2024-04-25T00:56:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "poolnum": 1, 2024-04-25T00:56:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: "poolname": ".mgr" 2024-04-25T00:56:16.452 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:16.452 INFO:teuthology.orchestra.run.smithi112.stdout:] 2024-04-25T00:56:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fafac0678c0 msgr2=0x7fafac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fafac0678c0 0x7fafac069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fafc00113e0 tx=0x7fafc000b040 comp rx=0 tx=0).stop 2024-04-25T00:56:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc80b3ce0 msgr2=0x7fafc8148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc80b3ce0 0x7fafc8148f90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fafb800c3a0 tx=0x7fafb800c860 comp rx=0 tx=0).stop 2024-04-25T00:56:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 shutdown_connections 2024-04-25T00:56:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fafac0678c0 0x7fafac069d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.454 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fafc8145200 0x7fafc813f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fafc80b3ce0 0x7fafc8148f90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 --2- 172.21.15.112:0/1695538904 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fafc80b23f0 0x7fafc813d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 >> 172.21.15.112:0/1695538904 conn(0x7fafc80b5980 msgr2=0x7fafc80726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:16.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 shutdown_connections 2024-04-25T00:56:16.455 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.451+0000 7fafcfc40640 1 -- 172.21.15.112:0/1695538904 wait complete. 2024-04-25T00:56:16.470 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:56:16.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:56:16.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8df185e640 1 Processor -- start 2024-04-25T00:56:16.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8df185e640 1 -- start start 2024-04-25T00:56:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8df185e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13f670 0x7f8dec150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.585 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8df185e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dec0b25c0 con 0x7f8dec0b4d90 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8df185e640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dec06f630 con 0x7f8dec0b21e0 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8df185e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8dec06f100 con 0x7f8dec13f670 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8dea7fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8dec0b4d90 0x7f8dec13ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41298/0 (socket says 172.21.15.112:41298) 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8dea7fc640 1 -- 172.21.15.112:0/674147351 learned_addr learned my addr 172.21.15.112:0/674147351 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8deb7fe640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13f670 0x7f8dec150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157927537 0 0) 0x7f8dec0b25c0 con 0x7f8dec0b4d90 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dd4003530 con 0x7f8dec0b4d90 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3982744690 0 0) 0x7f8dd4003530 con 0x7f8dec0b4d90 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13f670 msgr2=0x7f8dec150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13f670 0x7f8dec150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 >> v1:172.21.15.187:6789/0 conn(0x7f8dec0b21e0 legacy=0x7f8dec148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec06f350 con 0x7f8dec0b4d90 2024-04-25T00:56:16.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f8de0001680 con 0x7f8dec0b4d90 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 >> v1:172.21.15.112:6789/0 conn(0x7f8dec0b4d90 legacy=0x7f8dec13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dd40039f0 0x7f8dd4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13f670 0x7f8dd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd4004d50 0x7f8dd40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec06f100 con 0x7f8dd40039f0 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd400bb90 con 0x7f8dd4004d50 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd400bd90 con 0x7f8dec13f670 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8deaffd640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dd40039f0 0x7f8dd4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8deb7fe640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd4004d50 0x7f8dd40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8dea7fc640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13f670 0x7f8dd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8deaffd640 1 -- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13f670 msgr2=0x7f8dd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8deaffd640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13f670 0x7f8dd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8deaffd640 1 -- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd4004d50 msgr2=0x7f8dd40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:16.587 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8deaffd640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd4004d50 0x7f8dd40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.588 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.583+0000 7f8deaffd640 1 -- 172.21.15.112:0/674147351 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd4003530 con 0x7f8dd40039f0 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8deaffd640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dd40039f0 0x7f8dd4003e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc001780 tx=0x7f8ddc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32870b97b5f8d911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc0013a0 con 0x7f8dd40039f0 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ddc0052f0 con 0x7f8dd40039f0 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8de9ffb640 1 -- 172.21.15.112:0/674147351 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc003840 con 0x7f8dd40039f0 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- 172.21.15.112:0/674147351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dd40039f0 msgr2=0x7f8dd4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 --2- 172.21.15.112:0/674147351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dd40039f0 0x7f8dd4003e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc001780 tx=0x7f8ddc01c420 comp rx=0 tx=0).stop 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8deb7fe640 1 -- 172.21.15.112:0/674147351 reap_dead start 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- 172.21.15.112:0/674147351 shutdown_connections 2024-04-25T00:56:16.589 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- 172.21.15.112:0/674147351 >> 172.21.15.112:0/674147351 conn(0x7f8dec0b8930 msgr2=0x7f8dec0b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:16.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- 172.21.15.112:0/674147351 shutdown_connections 2024-04-25T00:56:16.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- 172.21.15.112:0/674147351 wait complete. 2024-04-25T00:56:16.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 Processor -- start 2024-04-25T00:56:16.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- start start 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd40039f0 0x7f8dec0b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dec071da0 0x7f8dec072200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13a340 0x7f8dec13a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec0b4b10 con 0x7f8dec071da0 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec0b4990 con 0x7f8dd40039f0 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8df185e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f8dec1512b0 con 0x7f8dec13a340 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8dea7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dec071da0 0x7f8dec072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8dea7fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dec071da0 0x7f8dec072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33666/0 (socket says 172.21.15.112:33666) 2024-04-25T00:56:16.592 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8dea7fc640 1 -- 172.21.15.112:0/3920556157 learned_addr learned my addr 172.21.15.112:0/3920556157 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8deaffd640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd40039f0 0x7f8dec0b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8deb7fe640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13a340 0x7f8dec13a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8dea7fc640 1 -- 172.21.15.112:0/3920556157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13a340 msgr2=0x7f8dec13a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8dea7fc640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13a340 0x7f8dec13a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8dea7fc640 1 -- 172.21.15.112:0/3920556157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd40039f0 msgr2=0x7f8dec0b7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8dea7fc640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd40039f0 0x7f8dec0b7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.587+0000 7f8dea7fc640 1 -- 172.21.15.112:0/3920556157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dec1400b0 con 0x7f8dec071da0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8dea7fc640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dec071da0 0x7f8dec072200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e780 tx=0x7f8de000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8dd3fff640 1 -- 172.21.15.112:0/3920556157 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de000c6c0 con 0x7f8dec071da0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8dd3fff640 1 -- 172.21.15.112:0/3920556157 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de0018070 con 0x7f8dec071da0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8dd3fff640 1 -- 172.21.15.112:0/3920556157 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de000ca80 con 0x7f8dec071da0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dec071450 con 0x7f8dec071da0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dec070bd0 con 0x7f8dec071da0 2024-04-25T00:56:16.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dec068260 con 0x7f8dec071da0 2024-04-25T00:56:16.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8dd3fff640 1 -- 172.21.15.112:0/3920556157 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8de000cc20 con 0x7f8dec071da0 2024-04-25T00:56:16.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8dd3fff640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8dd80678c0 0x7f8dd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:16.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8dd3fff640 1 -- 172.21.15.112:0/3920556157 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4857+0+0 (secure 0 0 0) 0x7f8de0085620 con 0x7f8dec071da0 2024-04-25T00:56:16.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8deaffd640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8dd80678c0 0x7f8dd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:16.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.591+0000 7f8deaffd640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8dd80678c0 0x7f8dd8069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc001750 tx=0x7f8ddc0011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:16.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.595+0000 7f8dd3fff640 1 -- 172.21.15.112:0/3920556157 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8de0056540 con 0x7f8dec071da0 2024-04-25T00:56:16.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:16.743+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8dec13c580 con 0x7f8dec071da0 2024-04-25T00:56:17.219 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8dd3fff640 1 -- 172.21.15.112:0/3920556157 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v26) ==== 138+0+0 (secure 0 0 0) 0x7f8de0055e40 con 0x7f8dec071da0 2024-04-25T00:56:17.219 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' created 2024-04-25T00:56:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8dd80678c0 msgr2=0x7f8dd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8dd80678c0 0x7f8dd8069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc001750 tx=0x7f8ddc0011a0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dec071da0 msgr2=0x7f8dec072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dec071da0 0x7f8dec072200 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8de000e780 tx=0x7f8de000ec40 comp rx=0 tx=0).stop 2024-04-25T00:56:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 shutdown_connections 2024-04-25T00:56:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f8dec13a340 0x7f8dec13a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8dec071da0 0x7f8dec072200 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f8dd80678c0 0x7f8dd8069d80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 --2- 172.21.15.112:0/3920556157 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f8dd40039f0 0x7f8dec0b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.219+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 >> 172.21.15.112:0/3920556157 conn(0x7f8dec0b8930 msgr2=0x7f8dec135a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.223+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 shutdown_connections 2024-04-25T00:56:17.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.223+0000 7f8df185e640 1 -- 172.21.15.112:0/3920556157 wait complete. 2024-04-25T00:56:17.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:56:17.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.367+0000 7f46ff135640 1 Processor -- start 2024-04-25T00:56:17.369 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.367+0000 7f46ff135640 1 -- start start 2024-04-25T00:56:17.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ff135640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f46f814b420 0x7f46f8152e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:17.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ff135640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f80b8fe0 con 0x7f46f81419d0 2024-04-25T00:56:17.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ff135640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46f809b810 con 0x7f46f80b8b00 2024-04-25T00:56:17.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ff135640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f46f809baf0 con 0x7f46f814b420 2024-04-25T00:56:17.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f46f81419d0 0x7f46f80bd460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41314/0 (socket says 172.21.15.112:41314) 2024-04-25T00:56:17.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 -- 172.21.15.112:0/455334622 learned_addr learned my addr 172.21.15.112:0/455334622 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:17.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1001620969 0 0) 0x7f46f80b8fe0 con 0x7f46f81419d0 2024-04-25T00:56:17.372 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46e0003530 con 0x7f46f81419d0 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fd6d2640 1 --2- 172.21.15.112:0/455334622 >> v2:172.21.15.187:3300/0 conn(0x7f46f814b420 0x7f46f8152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3400264500 0 0) 0x7f46e0003530 con 0x7f46f81419d0 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 >> v2:172.21.15.187:3300/0 conn(0x7f46f814b420 msgr2=0x7f46f8152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 --2- 172.21.15.112:0/455334622 >> v2:172.21.15.187:3300/0 conn(0x7f46f814b420 0x7f46f8152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 >> v1:172.21.15.187:6789/0 conn(0x7f46f80b8b00 legacy=0x7f46f814ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f809bdd0 con 0x7f46f81419d0 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f46f4001680 con 0x7f46f81419d0 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 >> v1:172.21.15.112:6789/0 conn(0x7f46f81419d0 legacy=0x7f46f80bd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46e0004810 0x7f46e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46e00052f0 0x7f46e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46f809baf0 con 0x7f46e00039c0 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f46e000c1c0 con 0x7f46e00052f0 2024-04-25T00:56:17.373 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f46e000c3c0 con 0x7f46e0004810 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fd6d2640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46e00052f0 0x7f46e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46effff640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46e0004810 0x7f46e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 -- 172.21.15.112:0/455334622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46e0004810 msgr2=0x7f46e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46e0004810 0x7f46e0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 -- 172.21.15.112:0/455334622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46e00052f0 msgr2=0x7f46e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46e00052f0 0x7f46e000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fd6d2640 1 -- 172.21.15.112:0/455334622 reap_dead start 2024-04-25T00:56:17.374 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 -- 172.21.15.112:0/455334622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e0003530 con 0x7f46e00039c0 2024-04-25T00:56:17.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46fced1640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46e0003de0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c210 tx=0x7f46f401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7a13abc75c1b3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:17.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4007d20 con 0x7f46e00039c0 2024-04-25T00:56:17.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f40a9070 con 0x7f46e00039c0 2024-04-25T00:56:17.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ef7fe640 1 -- 172.21.15.112:0/455334622 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4005d80 con 0x7f46e00039c0 2024-04-25T00:56:17.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ff135640 1 -- 172.21.15.112:0/455334622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 msgr2=0x7f46e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:17.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.371+0000 7f46ff135640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46e0003de0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f46f400c210 tx=0x7f46f401cc60 comp rx=0 tx=0).stop 2024-04-25T00:56:17.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- 172.21.15.112:0/455334622 shutdown_connections 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 --2- 172.21.15.112:0/455334622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46e0003de0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- 172.21.15.112:0/455334622 >> 172.21.15.112:0/455334622 conn(0x7f46f8067660 msgr2=0x7f46f80bcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- 172.21.15.112:0/455334622 shutdown_connections 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- 172.21.15.112:0/455334622 wait complete. 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 Processor -- start 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- start start 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46f8142a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46f81e06a0 0x7f46f81e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46f81e3750 0x7f46f81e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f46f8072500 con 0x7f46e00039c0 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f46f8072380 con 0x7f46f81e3750 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f46f8072680 con 0x7f46f81e06a0 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46f8142a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:17.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46f8142a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33688/0 (socket says 172.21.15.112:33688) 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 -- 172.21.15.112:0/3068289929 learned_addr learned my addr 172.21.15.112:0/3068289929 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fd6d2640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46f81e3750 0x7f46f81e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46effff640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46f81e06a0 0x7f46f81e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 -- 172.21.15.112:0/3068289929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46f81e06a0 msgr2=0x7f46f81e0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46f81e06a0 0x7f46f81e0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 -- 172.21.15.112:0/3068289929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46f81e3750 msgr2=0x7f46f81e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46f81e3750 0x7f46f81e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 -- 172.21.15.112:0/3068289929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f81e01e0 con 0x7f46e00039c0 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46fced1640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46f8142a90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f46f401ca90 tx=0x7f46e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ed7fa640 1 -- 172.21.15.112:0/3068289929 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4007240 con 0x7f46e00039c0 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ed7fa640 1 -- 172.21.15.112:0/3068289929 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f40073e0 con 0x7f46e00039c0 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f81e1060 con 0x7f46e00039c0 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ed7fa640 1 -- 172.21.15.112:0/3068289929 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f40047f0 con 0x7f46e00039c0 2024-04-25T00:56:17.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f8144430 con 0x7f46e00039c0 2024-04-25T00:56:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.375+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f8144670 con 0x7f46e00039c0 2024-04-25T00:56:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.379+0000 7f46ed7fa640 1 -- 172.21.15.112:0/3068289929 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46f4004990 con 0x7f46e00039c0 2024-04-25T00:56:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.379+0000 7f46ed7fa640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46e40678f0 0x7f46e4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.379+0000 7f46ed7fa640 1 -- 172.21.15.112:0/3068289929 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5227+0+0 (secure 0 0 0) 0x7f46f411f610 con 0x7f46e00039c0 2024-04-25T00:56:17.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.379+0000 7f46effff640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46e40678f0 0x7f46e4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:17.381 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.379+0000 7f46effff640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46e40678f0 0x7f46e4069db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46e8000c10 tx=0x7f46e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:17.383 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.383+0000 7f46ed7fa640 1 -- 172.21.15.112:0/3068289929 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f46f81e1060 con 0x7f46e00039c0 2024-04-25T00:56:17.529 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:17.527+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f46f8068260 con 0x7f46e00039c0 2024-04-25T00:56:18.233 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.231+0000 7f46ed7fa640 1 -- 172.21.15.112:0/3068289929 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v27) ==== 170+0+0 (secure 0 0 0) 0x7f46f81e01e0 con 0x7f46e00039c0 2024-04-25T00:56:18.233 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' created 2024-04-25T00:56:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46e40678f0 msgr2=0x7f46e4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46e40678f0 0x7f46e4069db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f46e8000c10 tx=0x7f46e8009040 comp rx=0 tx=0).stop 2024-04-25T00:56:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 msgr2=0x7f46f8142a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46f8142a90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f46f401ca90 tx=0x7f46e0005840 comp rx=0 tx=0).stop 2024-04-25T00:56:18.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 shutdown_connections 2024-04-25T00:56:18.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f46e40678f0 0x7f46e4069db0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f46f81e3750 0x7f46f81e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f46f81e06a0 0x7f46f81e0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 --2- 172.21.15.112:0/3068289929 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f46e00039c0 0x7f46f8142a90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 >> 172.21.15.112:0/3068289929 conn(0x7f46f8067660 msgr2=0x7f46f8069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:18.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 shutdown_connections 2024-04-25T00:56:18.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.235+0000 7f46ff135640 1 -- 172.21.15.112:0/3068289929 wait complete. 2024-04-25T00:56:18.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:56:18.389 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.387+0000 7f6043f71640 1 Processor -- start 2024-04-25T00:56:18.390 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.387+0000 7f6043f71640 1 -- start start 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6043f71640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f603c06c170 0x7f603c0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6043f71640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b4020 0x7f603c14ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6043f71640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603c06c550 con 0x7f603c0b34b0 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6043f71640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f603c0ae8a0 con 0x7f603c06c170 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6043f71640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f603c0ae220 con 0x7f603c0b4020 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6041d0d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f603c06c170 0x7f603c0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f604250e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b4020 0x7f603c14ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6041d0d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f603c06c170 0x7f603c0aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:33562/0 (socket says 172.21.15.112:33562) 2024-04-25T00:56:18.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6041d0d640 1 -- 172.21.15.112:0/3259221503 learned_addr learned my addr 172.21.15.112:0/3259221503 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:18.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6041d0d640 1 -- 172.21.15.112:0/3259221503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b4020 msgr2=0x7f603c14ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:18.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6041d0d640 1 --2- 172.21.15.112:0/3259221503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b4020 0x7f603c14ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6041d0d640 1 -- 172.21.15.112:0/3259221503 >> v1:172.21.15.187:6789/0 conn(0x7f603c0b34b0 legacy=0x7f603c0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:18.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6041d0d640 1 -- 172.21.15.112:0/3259221503 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c0b0570 con 0x7f603c06c170 2024-04-25T00:56:18.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6041d0d640 1 --2- 172.21.15.112:0/3259221503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c06c170 0x7f603c0aedd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6034009e50 tx=0x7f603401c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8eb2c0e2b89a719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:18.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.391+0000 7f6040d0b640 1 -- 172.21.15.112:0/3259221503 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340a5750 con 0x7f603c06c170 2024-04-25T00:56:18.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6040d0b640 1 -- 172.21.15.112:0/3259221503 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60340a58f0 con 0x7f603c06c170 2024-04-25T00:56:18.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6040d0b640 1 -- 172.21.15.112:0/3259221503 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340a5cc0 con 0x7f603c06c170 2024-04-25T00:56:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- 172.21.15.112:0/3259221503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c06c170 msgr2=0x7f603c0aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 --2- 172.21.15.112:0/3259221503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c06c170 0x7f603c0aedd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6034009e50 tx=0x7f603401c9a0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- 172.21.15.112:0/3259221503 shutdown_connections 2024-04-25T00:56:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 --2- 172.21.15.112:0/3259221503 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b4020 0x7f603c14ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 --2- 172.21.15.112:0/3259221503 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c06c170 0x7f603c0aedd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- 172.21.15.112:0/3259221503 >> 172.21.15.112:0/3259221503 conn(0x7f603c0b8c80 msgr2=0x7f603c0b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:18.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- 172.21.15.112:0/3259221503 shutdown_connections 2024-04-25T00:56:18.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- 172.21.15.112:0/3259221503 wait complete. 2024-04-25T00:56:18.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 Processor -- start 2024-04-25T00:56:18.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- start start 2024-04-25T00:56:18.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c06c170 0x7f603c1dd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:18.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b34b0 0x7f603c1dde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c0b4020 0x7f603c1e1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0b51f0 con 0x7f603c06c170 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f603c0b2ea0 con 0x7f603c0b4020 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f603c0b5370 con 0x7f603c0b34b0 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c06c170 0x7f603c1dd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c06c170 0x7f603c1dd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33698/0 (socket says 172.21.15.112:33698) 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 -- 172.21.15.112:0/3939457559 learned_addr learned my addr 172.21.15.112:0/3939457559 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f604250e640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c0b4020 0x7f603c1e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f604150c640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b34b0 0x7f603c1dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 -- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b34b0 msgr2=0x7f603c1dde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b34b0 0x7f603c1dde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 -- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c0b4020 msgr2=0x7f603c1e1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c0b4020 0x7f603c1e1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:18.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 -- 172.21.15.112:0/3939457559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c09aa30 con 0x7f603c06c170 2024-04-25T00:56:18.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f604250e640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c0b4020 0x7f603c1e1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:18.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6041d0d640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c06c170 0x7f603c1dd940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f603401a5d0 tx=0x7f6034019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:18.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f603a7fc640 1 -- 172.21.15.112:0/3939457559 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340bc020 con 0x7f603c06c170 2024-04-25T00:56:18.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f603a7fc640 1 -- 172.21.15.112:0/3939457559 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60340b8670 con 0x7f603c06c170 2024-04-25T00:56:18.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c09b5c0 con 0x7f603c06c170 2024-04-25T00:56:18.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f603a7fc640 1 -- 172.21.15.112:0/3939457559 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340b8a20 con 0x7f603c06c170 2024-04-25T00:56:18.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.395+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c1de7d0 con 0x7f603c06c170 2024-04-25T00:56:18.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.399+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c04f5f0 con 0x7f603c06c170 2024-04-25T00:56:18.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.399+0000 7f603a7fc640 1 -- 172.21.15.112:0/3939457559 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f60340bf020 con 0x7f603c06c170 2024-04-25T00:56:18.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.399+0000 7f603a7fc640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f60280678c0 0x7f6028069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:18.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.399+0000 7f603a7fc640 1 -- 172.21.15.112:0/3939457559 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5609+0+0 (secure 0 0 0) 0x7f6034120b60 con 0x7f603c06c170 2024-04-25T00:56:18.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.399+0000 7f604150c640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f60280678c0 0x7f6028069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:18.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.399+0000 7f604150c640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f60280678c0 0x7f6028069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f603c14e680 tx=0x7f6030009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:18.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.403+0000 7f603a7fc640 1 -- 172.21.15.112:0/3939457559 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f60340c6030 con 0x7f603c06c170 2024-04-25T00:56:18.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:18.583+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f603c0b6ac0 con 0x7f603c06c170 2024-04-25T00:56:19.264 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.263+0000 7f603a7fc640 1 -- 172.21.15.112:0/3939457559 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f6034016620 con 0x7f603c06c170 2024-04-25T00:56:19.265 INFO:teuthology.orchestra.run.smithi112.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T00:56:19.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f60280678c0 msgr2=0x7f6028069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f60280678c0 0x7f6028069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f603c14e680 tx=0x7f6030009040 comp rx=0 tx=0).stop 2024-04-25T00:56:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c06c170 msgr2=0x7f603c1dd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c06c170 0x7f603c1dd940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f603401a5d0 tx=0x7f6034019740 comp rx=0 tx=0).stop 2024-04-25T00:56:19.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 shutdown_connections 2024-04-25T00:56:19.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f60280678c0 0x7f6028069d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f603c0b4020 0x7f603c1e1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f603c0b34b0 0x7f603c1dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 --2- 172.21.15.112:0/3939457559 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f603c06c170 0x7f603c1dd940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 >> 172.21.15.112:0/3939457559 conn(0x7f603c0b8c80 msgr2=0x7f603c0b6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:19.277 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 shutdown_connections 2024-04-25T00:56:19.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.275+0000 7f6043f71640 1 -- 172.21.15.112:0/3939457559 wait complete. 2024-04-25T00:56:19.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.415+0000 7f0698da4640 1 Processor -- start 2024-04-25T00:56:19.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.415+0000 7f0698da4640 1 -- start start 2024-04-25T00:56:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f0698da4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f0694067120 0x7f06940b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f0698da4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f06940b52f0 0x7f069406d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f0698da4640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06940b5f60 con 0x7f0694072af0 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f0698da4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0694070cb0 con 0x7f06940b52f0 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f0698da4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0694071420 con 0x7f0694067120 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f068bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f06940b52f0 0x7f069406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f068bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f06940b52f0 0x7f069406d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33702/0 (socket says 172.21.15.112:33702) 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f068bfff640 1 -- 172.21.15.112:0/3343075048 learned_addr learned my addr 172.21.15.112:0/3343075048 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f069259c640 1 --2- 172.21.15.112:0/3343075048 >> v2:172.21.15.187:3300/0 conn(0x7f0694067120 0x7f06940b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f068bfff640 1 -- 172.21.15.112:0/3343075048 >> v2:172.21.15.187:3300/0 conn(0x7f0694067120 msgr2=0x7f06940b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f068bfff640 1 --2- 172.21.15.112:0/3343075048 >> v2:172.21.15.187:3300/0 conn(0x7f0694067120 0x7f06940b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f068bfff640 1 -- 172.21.15.112:0/3343075048 >> v1:172.21.15.187:6789/0 conn(0x7f0694072af0 legacy=0x7f069406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:19.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f068bfff640 1 -- 172.21.15.112:0/3343075048 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06940b4890 con 0x7f06940b52f0 2024-04-25T00:56:19.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.419+0000 7f068bfff640 1 --2- 172.21.15.112:0/3343075048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f069406d180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f06940b0d20 tx=0x7f067c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468a76d9bed8f653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:19.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0691d9b640 1 -- 172.21.15.112:0/3343075048 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0b2070 con 0x7f06940b52f0 2024-04-25T00:56:19.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0691d9b640 1 -- 172.21.15.112:0/3343075048 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067c0a9070 con 0x7f06940b52f0 2024-04-25T00:56:19.422 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0691d9b640 1 -- 172.21.15.112:0/3343075048 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067c0ad880 con 0x7f06940b52f0 2024-04-25T00:56:19.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- 172.21.15.112:0/3343075048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 msgr2=0x7f069406d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 --2- 172.21.15.112:0/3343075048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f069406d180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f06940b0d20 tx=0x7f067c01c450 comp rx=0 tx=0).stop 2024-04-25T00:56:19.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- 172.21.15.112:0/3343075048 shutdown_connections 2024-04-25T00:56:19.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 --2- 172.21.15.112:0/3343075048 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f069406d180 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 --2- 172.21.15.112:0/3343075048 >> v2:172.21.15.187:3300/0 conn(0x7f0694067120 0x7f06940b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- 172.21.15.112:0/3343075048 >> 172.21.15.112:0/3343075048 conn(0x7f069406b3e0 msgr2=0x7f069406b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:19.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- 172.21.15.112:0/3343075048 shutdown_connections 2024-04-25T00:56:19.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- 172.21.15.112:0/3343075048 wait complete. 2024-04-25T00:56:19.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 Processor -- start 2024-04-25T00:56:19.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- start start 2024-04-25T00:56:19.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0694067120 0x7f0694147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0694072af0 0x7f06941e0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f06941e1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f069406df50 con 0x7f06940b52f0 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f069406ddd0 con 0x7f0694067120 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0694152600 con 0x7f0694072af0 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f06941e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f06941e1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33706/0 (socket says 172.21.15.112:33706) 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 -- 172.21.15.112:0/547620213 learned_addr learned my addr 172.21.15.112:0/547620213 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f069259c640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0694067120 0x7f0694147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f068bfff640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0694072af0 0x7f06941e0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 -- 172.21.15.112:0/547620213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0694072af0 msgr2=0x7f06941e0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0694072af0 0x7f06941e0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 -- 172.21.15.112:0/547620213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0694067120 msgr2=0x7f0694147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0694067120 0x7f0694147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 -- 172.21.15.112:0/547620213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0694144740 con 0x7f06940b52f0 2024-04-25T00:56:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0692d9d640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f06941e1130 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c780 tx=0x7f068c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f068affd640 1 -- 172.21.15.112:0/547620213 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c016020 con 0x7f06940b52f0 2024-04-25T00:56:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f068affd640 1 -- 172.21.15.112:0/547620213 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068c00a9f0 con 0x7f06940b52f0 2024-04-25T00:56:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0694144a20 con 0x7f06940b52f0 2024-04-25T00:56:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f068affd640 1 -- 172.21.15.112:0/547620213 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c00ae20 con 0x7f06940b52f0 2024-04-25T00:56:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.423+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069413b510 con 0x7f06940b52f0 2024-04-25T00:56:19.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.427+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069404f580 con 0x7f06940b52f0 2024-04-25T00:56:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.427+0000 7f068affd640 1 -- 172.21.15.112:0/547620213 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f068c019020 con 0x7f06940b52f0 2024-04-25T00:56:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.427+0000 7f068affd640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06740678c0 0x7f0674069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.427+0000 7f069259c640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06740678c0 0x7f0674069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.427+0000 7f068affd640 1 -- 172.21.15.112:0/547620213 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f068c085930 con 0x7f06940b52f0 2024-04-25T00:56:19.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.427+0000 7f069259c640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06740678c0 0x7f0674069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06941de720 tx=0x7f0684008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:19.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.431+0000 7f068affd640 1 -- 172.21.15.112:0/547620213 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f068c020070 con 0x7f06940b52f0 2024-04-25T00:56:19.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.591+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f069413bd00 con 0x7f06940b52f0 2024-04-25T00:56:19.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f068affd640 1 -- 172.21.15.112:0/547620213 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+16730 (secure 0 0 0) 0x7f068c056460 con 0x7f06940b52f0 2024-04-25T00:56:19.596 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:19.596 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:18.588426+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:56:16.783920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:56:17.569481+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06740678c0 msgr2=0x7f0674069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06740678c0 0x7f0674069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06941de720 tx=0x7f0684008040 comp rx=0 tx=0).stop 2024-04-25T00:56:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 msgr2=0x7f06941e1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f06941e1130 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c780 tx=0x7f068c00cc40 comp rx=0 tx=0).stop 2024-04-25T00:56:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 shutdown_connections 2024-04-25T00:56:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06940b52f0 0x7f06941e1130 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0694072af0 0x7f06941e0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06740678c0 0x7f0674069d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 --2- 172.21.15.112:0/547620213 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f0694067120 0x7f0694147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.595+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 >> 172.21.15.112:0/547620213 conn(0x7f069406b3e0 msgr2=0x7f0694071e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:19.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.599+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 shutdown_connections 2024-04-25T00:56:19.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.599+0000 7f0698da4640 1 -- 172.21.15.112:0/547620213 wait complete. 2024-04-25T00:56:19.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:19.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.731+0000 7f80d751c640 1 Processor -- start 2024-04-25T00:56:19.734 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.731+0000 7f80d751c640 1 -- start start 2024-04-25T00:56:19.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.731+0000 7f80d751c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f80d00722d0 0x7f80d0072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d009c270 0x7f80d0147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f80d0147650 0x7f80d0148240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f80d00baaf0 con 0x7f80d00722d0 2024-04-25T00:56:19.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f80d00bbec0 con 0x7f80d0147650 2024-04-25T00:56:19.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80d0073100 con 0x7f80d009c270 2024-04-25T00:56:19.735 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f80d00722d0 0x7f80d0072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f80d00722d0 0x7f80d0072bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33720/0 (socket says 172.21.15.112:33720) 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 -- 172.21.15.112:0/1297141477 learned_addr learned my addr 172.21.15.112:0/1297141477 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d5ab9640 1 --2- 172.21.15.112:0/1297141477 >> v2:172.21.15.187:3300/0 conn(0x7f80d0147650 0x7f80d0148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d4ab7640 1 --2- 172.21.15.112:0/1297141477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d009c270 0x7f80d0147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 -- 172.21.15.112:0/1297141477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d009c270 msgr2=0x7f80d0147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 --2- 172.21.15.112:0/1297141477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d009c270 0x7f80d0147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 -- 172.21.15.112:0/1297141477 >> v2:172.21.15.187:3300/0 conn(0x7f80d0147650 msgr2=0x7f80d0148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 --2- 172.21.15.112:0/1297141477 >> v2:172.21.15.187:3300/0 conn(0x7f80d0147650 0x7f80d0148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 -- 172.21.15.112:0/1297141477 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d00bc5c0 con 0x7f80d00722d0 2024-04-25T00:56:19.736 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d5ab9640 1 --2- 172.21.15.112:0/1297141477 >> v2:172.21.15.187:3300/0 conn(0x7f80d0147650 0x7f80d0148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d52b8640 1 --2- 172.21.15.112:0/1297141477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d00722d0 0x7f80d0072bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f80d00ba0f0 tx=0x7f80c001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bada5047bb041b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80c7fff640 1 -- 172.21.15.112:0/1297141477 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00a7070 con 0x7f80d00722d0 2024-04-25T00:56:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80c7fff640 1 -- 172.21.15.112:0/1297141477 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c00b0070 con 0x7f80d00722d0 2024-04-25T00:56:19.737 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80c7fff640 1 -- 172.21.15.112:0/1297141477 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00ab890 con 0x7f80d00722d0 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 -- 172.21.15.112:0/1297141477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d00722d0 msgr2=0x7f80d0072bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 --2- 172.21.15.112:0/1297141477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d00722d0 0x7f80d0072bc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f80d00ba0f0 tx=0x7f80c001c4f0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 -- 172.21.15.112:0/1297141477 shutdown_connections 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 --2- 172.21.15.112:0/1297141477 >> v2:172.21.15.187:3300/0 conn(0x7f80d0147650 0x7f80d0148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 --2- 172.21.15.112:0/1297141477 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d009c270 0x7f80d0147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 --2- 172.21.15.112:0/1297141477 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d00722d0 0x7f80d0072bc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 -- 172.21.15.112:0/1297141477 >> 172.21.15.112:0/1297141477 conn(0x7f80d009e8c0 msgr2=0x7f80d009ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 -- 172.21.15.112:0/1297141477 shutdown_connections 2024-04-25T00:56:19.738 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.735+0000 7f80d751c640 1 -- 172.21.15.112:0/1297141477 wait complete. 2024-04-25T00:56:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 Processor -- start 2024-04-25T00:56:19.739 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 -- start start 2024-04-25T00:56:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d00722d0 0x7f80d014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d009c270 0x7f80d01dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d0147650 0x7f80d01e0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f80d014b380 con 0x7f80d009c270 2024-04-25T00:56:19.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f80d009ac50 con 0x7f80d0147650 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f80d014b500 con 0x7f80d00722d0 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d009c270 0x7f80d01dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d009c270 0x7f80d01dfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33724/0 (socket says 172.21.15.112:33724) 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 -- 172.21.15.112:0/2373126337 learned_addr learned my addr 172.21.15.112:0/2373126337 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d52b8640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d00722d0 0x7f80d014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d5ab9640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d0147650 0x7f80d01e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 -- 172.21.15.112:0/2373126337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d00722d0 msgr2=0x7f80d014c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d00722d0 0x7f80d014c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 -- 172.21.15.112:0/2373126337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d0147650 msgr2=0x7f80d01e0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d0147650 0x7f80d01e0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 -- 172.21.15.112:0/2373126337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d014d080 con 0x7f80d009c270 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d4ab7640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d009c270 0x7f80d01dfed0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c7f0 tx=0x7f80cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:19.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80c5ffb640 1 -- 172.21.15.112:0/2373126337 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc015070 con 0x7f80d009c270 2024-04-25T00:56:19.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80c5ffb640 1 -- 172.21.15.112:0/2373126337 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80cc007820 con 0x7f80d009c270 2024-04-25T00:56:19.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d014e0d0 con 0x7f80d009c270 2024-04-25T00:56:19.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80c5ffb640 1 -- 172.21.15.112:0/2373126337 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80cc007bf0 con 0x7f80d009c270 2024-04-25T00:56:19.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d014ddf0 con 0x7f80d009c270 2024-04-25T00:56:19.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d014d280 con 0x7f80d009c270 2024-04-25T00:56:19.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.739+0000 7f80c5ffb640 1 -- 172.21.15.112:0/2373126337 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f80cc018070 con 0x7f80d009c270 2024-04-25T00:56:19.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.743+0000 7f80c5ffb640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f80b40678f0 0x7f80b4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:19.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.743+0000 7f80c5ffb640 1 -- 172.21.15.112:0/2373126337 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f80cc085bd0 con 0x7f80d009c270 2024-04-25T00:56:19.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.743+0000 7f80d52b8640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f80b40678f0 0x7f80b4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:19.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.743+0000 7f80d52b8640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f80b40678f0 0x7f80b4069db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80c00af000 tx=0x7f80c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:19.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.747+0000 7f80c5ffb640 1 -- 172.21.15.112:0/2373126337 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f80d014d280 con 0x7f80d009c270 2024-04-25T00:56:19.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.907+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80d0147160 con 0x7f80d009c270 2024-04-25T00:56:19.912 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.911+0000 7f80c5ffb640 1 -- 172.21.15.112:0/2373126337 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+16730 (secure 0 0 0) 0x7f80cc0567a0 con 0x7f80d009c270 2024-04-25T00:56:19.913 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:19.913 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":28,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:18.588426+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:56:16.783920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:56:17.569481+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f80b40678f0 msgr2=0x7f80b4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f80b40678f0 0x7f80b4069db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f80c00af000 tx=0x7f80c0008040 comp rx=0 tx=0).stop 2024-04-25T00:56:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d009c270 msgr2=0x7f80d01dfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d009c270 0x7f80d01dfed0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c7f0 tx=0x7f80cc00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 shutdown_connections 2024-04-25T00:56:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f80b40678f0 0x7f80b4069db0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f80d0147650 0x7f80d01e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f80d009c270 0x7f80d01dfed0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 --2- 172.21.15.112:0/2373126337 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f80d00722d0 0x7f80d014c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:19.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 >> 172.21.15.112:0/2373126337 conn(0x7f80d009e8c0 msgr2=0x7f80d00b8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:19.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 shutdown_connections 2024-04-25T00:56:19.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:19.915+0000 7f80d751c640 1 -- 172.21.15.112:0/2373126337 wait complete. 2024-04-25T00:56:19.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:56:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27c10b4640 1 Processor -- start 2024-04-25T00:56:20.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27c10b4640 1 -- start start 2024-04-25T00:56:20.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27c10b4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f27bc0b9c60 0x7f27bc0b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27c10b4640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f27bc0b5800 0x7f27bc140f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27c10b4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc1414d0 0x7f27bc1473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27c10b4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f27bc0b6b70 con 0x7f27bc0b9c60 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27c10b4640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f27bc0b4200 con 0x7f27bc0b5800 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27c10b4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f27bc0b44e0 con 0x7f27bc1414d0 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27bad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f27bc0b9c60 0x7f27bc0b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27bad9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f27bc0b9c60 0x7f27bc0b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33732/0 (socket says 172.21.15.112:33732) 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27bad9d640 1 -- 172.21.15.112:0/1678239014 learned_addr learned my addr 172.21.15.112:0/1678239014 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27bad9d640 1 -- 172.21.15.112:0/1678239014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc1414d0 msgr2=0x7f27bc1473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27bad9d640 1 --2- 172.21.15.112:0/1678239014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc1414d0 0x7f27bc1473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:20.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27ba59c640 1 --2- 172.21.15.112:0/1678239014 >> v2:172.21.15.187:3300/0 conn(0x7f27bc0b5800 0x7f27bc140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:20.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27bad9d640 1 -- 172.21.15.112:0/1678239014 >> v2:172.21.15.187:3300/0 conn(0x7f27bc0b5800 msgr2=0x7f27bc140f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:20.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27bad9d640 1 --2- 172.21.15.112:0/1678239014 >> v2:172.21.15.187:3300/0 conn(0x7f27bc0b5800 0x7f27bc140f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:20.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.051+0000 7f27bad9d640 1 -- 172.21.15.112:0/1678239014 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc0b47c0 con 0x7f27bc0b9c60 2024-04-25T00:56:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27bad9d640 1 --2- 172.21.15.112:0/1678239014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc0b9c60 0x7f27bc0b52c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0bdcf0 tx=0x7f27a401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e6517e6e49a209 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27b9d9b640 1 -- 172.21.15.112:0/1678239014 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40b0070 con 0x7f27bc0b9c60 2024-04-25T00:56:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27b9d9b640 1 -- 172.21.15.112:0/1678239014 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27a40ab600 con 0x7f27bc0b9c60 2024-04-25T00:56:20.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27b9d9b640 1 -- 172.21.15.112:0/1678239014 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a40ab9d0 con 0x7f27bc0b9c60 2024-04-25T00:56:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- 172.21.15.112:0/1678239014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc0b9c60 msgr2=0x7f27bc0b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 --2- 172.21.15.112:0/1678239014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc0b9c60 0x7f27bc0b52c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0bdcf0 tx=0x7f27a401c400 comp rx=0 tx=0).stop 2024-04-25T00:56:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- 172.21.15.112:0/1678239014 shutdown_connections 2024-04-25T00:56:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 --2- 172.21.15.112:0/1678239014 >> v2:172.21.15.187:3300/0 conn(0x7f27bc0b5800 0x7f27bc140f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 --2- 172.21.15.112:0/1678239014 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc1414d0 0x7f27bc1473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 --2- 172.21.15.112:0/1678239014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc0b9c60 0x7f27bc0b52c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- 172.21.15.112:0/1678239014 >> 172.21.15.112:0/1678239014 conn(0x7f27bc067120 msgr2=0x7f27bc067530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:20.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- 172.21.15.112:0/1678239014 shutdown_connections 2024-04-25T00:56:20.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- 172.21.15.112:0/1678239014 wait complete. 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 Processor -- start 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- start start 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27bc0b5800 0x7f27bc1498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc0b9c60 0x7f27bc145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc1414d0 0x7f27bc145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc14b480 con 0x7f27bc1414d0 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc14b300 con 0x7f27bc0b5800 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27c10b4640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f27bc14b600 con 0x7f27bc0b9c60 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27bb59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc1414d0 0x7f27bc145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27bb59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc1414d0 0x7f27bc145940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33748/0 (socket says 172.21.15.112:33748) 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27bb59e640 1 -- 172.21.15.112:0/2824521622 learned_addr learned my addr 172.21.15.112:0/2824521622 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27ba59c640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc0b9c60 0x7f27bc145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:20.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.055+0000 7f27bad9d640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27bc0b5800 0x7f27bc1498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27bb59e640 1 -- 172.21.15.112:0/2824521622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc0b9c60 msgr2=0x7f27bc145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27bb59e640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc0b9c60 0x7f27bc145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27bb59e640 1 -- 172.21.15.112:0/2824521622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27bc0b5800 msgr2=0x7f27bc1498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27bb59e640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27bc0b5800 0x7f27bc1498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27bb59e640 1 -- 172.21.15.112:0/2824521622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc13bd10 con 0x7f27bc1414d0 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27bb59e640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc1414d0 0x7f27bc145940 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b9e0 tx=0x7f27b400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27b37fe640 1 -- 172.21.15.112:0/2824521622 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4019070 con 0x7f27bc1414d0 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27b37fe640 1 -- 172.21.15.112:0/2824521622 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b400cb20 con 0x7f27bc1414d0 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc13a0f0 con 0x7f27bc1414d0 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27b37fe640 1 -- 172.21.15.112:0/2824521622 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b4014440 con 0x7f27bc1414d0 2024-04-25T00:56:20.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc13b9a0 con 0x7f27bc1414d0 2024-04-25T00:56:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc145160 con 0x7f27bc1414d0 2024-04-25T00:56:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27b37fe640 1 -- 172.21.15.112:0/2824521622 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f27b401c070 con 0x7f27bc1414d0 2024-04-25T00:56:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27b37fe640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f279c0678c0 0x7f279c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27b37fe640 1 -- 172.21.15.112:0/2824521622 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5709+0+0 (secure 0 0 0) 0x7f27b4085980 con 0x7f27bc1414d0 2024-04-25T00:56:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27bad9d640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f279c0678c0 0x7f279c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:20.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.059+0000 7f27bad9d640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f279c0678c0 0x7f279c069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27a40ad000 tx=0x7f27a40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:20.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.063+0000 7f27b37fe640 1 -- 172.21.15.112:0/2824521622 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27b40564c0 con 0x7f27bc1414d0 2024-04-25T00:56:20.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:20.259+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f27bc0b6890 con 0x7f27bc1414d0 2024-04-25T00:56:21.247 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.247+0000 7f27b37fe640 1 -- 172.21.15.112:0/2824521622 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f27b4055dc0 con 0x7f27bc1414d0 2024-04-25T00:56:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f279c0678c0 msgr2=0x7f279c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:21.251 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f279c0678c0 0x7f279c069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27a40ad000 tx=0x7f27a40a6000 comp rx=0 tx=0).stop 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc1414d0 msgr2=0x7f27bc145940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc1414d0 0x7f27bc145940 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f27b400b9e0 tx=0x7f27b400bea0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 shutdown_connections 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f27bc1414d0 0x7f27bc145940 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f27bc0b9c60 0x7f27bc145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f279c0678c0 0x7f279c069d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 --2- 172.21.15.112:0/2824521622 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f27bc0b5800 0x7f27bc1498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 >> 172.21.15.112:0/2824521622 conn(0x7f27bc067120 msgr2=0x7f27bc132240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 shutdown_connections 2024-04-25T00:56:21.252 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.251+0000 7f27c10b4640 1 -- 172.21.15.112:0/2824521622 wait complete. 2024-04-25T00:56:21.270 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T00:56:21.270 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T00:56:21.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f90881cb640 1 Processor -- start 2024-04-25T00:56:21.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f90881cb640 1 -- start start 2024-04-25T00:56:21.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f90881cb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f90800b9030 0x7f908009acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:21.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f90881cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908014fcd0 0x7f90801520c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:21.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f90881cb640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90800b9410 con 0x7f908009b800 2024-04-25T00:56:21.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f90881cb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f90800b9990 con 0x7f90800b9030 2024-04-25T00:56:21.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f90881cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f90800baa90 con 0x7f908014fcd0 2024-04-25T00:56:21.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9086768640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908014fcd0 0x7f90801520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:21.392 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9085f67640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f90800b9030 0x7f908009acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:21.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9086768640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908014fcd0 0x7f90801520c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:52328/0 (socket says 172.21.15.112:52328) 2024-04-25T00:56:21.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9086768640 1 -- 172.21.15.112:0/1243503995 learned_addr learned my addr 172.21.15.112:0/1243503995 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:21.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9086768640 1 -- 172.21.15.112:0/1243503995 >> v2:172.21.15.187:3300/0 conn(0x7f90800b9030 msgr2=0x7f908009acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:21.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9086768640 1 --2- 172.21.15.112:0/1243503995 >> v2:172.21.15.187:3300/0 conn(0x7f90800b9030 0x7f908009acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9086768640 1 -- 172.21.15.112:0/1243503995 >> v1:172.21.15.187:6789/0 conn(0x7f908009b800 legacy=0x7f9080141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:21.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9086768640 1 -- 172.21.15.112:0/1243503995 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90800bad70 con 0x7f908014fcd0 2024-04-25T00:56:21.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.391+0000 7f9085f67640 1 --2- 172.21.15.112:0/1243503995 >> v2:172.21.15.187:3300/0 conn(0x7f90800b9030 0x7f908009acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:56:21.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f9086768640 1 --2- 172.21.15.112:0/1243503995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908014fcd0 0x7f90801520c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90800b9b10 tx=0x7f907801c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b43a9dba17c2238 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:21.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f9084f65640 1 -- 172.21.15.112:0/1243503995 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90780a56d0 con 0x7f908014fcd0 2024-04-25T00:56:21.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f9084f65640 1 -- 172.21.15.112:0/1243503995 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90780a5870 con 0x7f908014fcd0 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f9084f65640 1 -- 172.21.15.112:0/1243503995 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90780a5c40 con 0x7f908014fcd0 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 -- 172.21.15.112:0/1243503995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908014fcd0 msgr2=0x7f90801520c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 --2- 172.21.15.112:0/1243503995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908014fcd0 0x7f90801520c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90800b9b10 tx=0x7f907801c980 comp rx=0 tx=0).stop 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 -- 172.21.15.112:0/1243503995 shutdown_connections 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 --2- 172.21.15.112:0/1243503995 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f908014fcd0 0x7f90801520c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 --2- 172.21.15.112:0/1243503995 >> v2:172.21.15.187:3300/0 conn(0x7f90800b9030 0x7f908009acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 -- 172.21.15.112:0/1243503995 >> 172.21.15.112:0/1243503995 conn(0x7f908009eaf0 msgr2=0x7f908009ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 -- 172.21.15.112:0/1243503995 shutdown_connections 2024-04-25T00:56:21.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 -- 172.21.15.112:0/1243503995 wait complete. 2024-04-25T00:56:21.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 Processor -- start 2024-04-25T00:56:21.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.395+0000 7f90881cb640 1 -- start start 2024-04-25T00:56:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908009b800 0x7f90801e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90800b9030 0x7f9080150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:21.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908014fcd0 0x7f90801511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f908009b370 con 0x7f908014fcd0 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f908009b1f0 con 0x7f908009b800 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9080141a60 con 0x7f90800b9030 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908014fcd0 0x7f90801511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908014fcd0 0x7f90801511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33756/0 (socket says 172.21.15.112:33756) 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 -- 172.21.15.112:0/604678447 learned_addr learned my addr 172.21.15.112:0/604678447 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9085f67640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908009b800 0x7f90801e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9085766640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90800b9030 0x7f9080150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 -- 172.21.15.112:0/604678447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90800b9030 msgr2=0x7f9080150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90800b9030 0x7f9080150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 -- 172.21.15.112:0/604678447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908009b800 msgr2=0x7f90801e33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908009b800 0x7f90801e33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:21.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 -- 172.21.15.112:0/604678447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90801501c0 con 0x7f908014fcd0 2024-04-25T00:56:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f9086768640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908014fcd0 0x7f90801511e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f907801a5b0 tx=0x7f9078019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90767fc640 1 -- 172.21.15.112:0/604678447 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90780bc020 con 0x7f908014fcd0 2024-04-25T00:56:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90767fc640 1 -- 172.21.15.112:0/604678447 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90780b8670 con 0x7f908014fcd0 2024-04-25T00:56:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90801504a0 con 0x7f908014fcd0 2024-04-25T00:56:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90767fc640 1 -- 172.21.15.112:0/604678447 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90780b8a40 con 0x7f908014fcd0 2024-04-25T00:56:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90800b43c0 con 0x7f908014fcd0 2024-04-25T00:56:21.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.399+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f908009e690 con 0x7f908014fcd0 2024-04-25T00:56:21.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.403+0000 7f90767fc640 1 -- 172.21.15.112:0/604678447 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f90780bf020 con 0x7f908014fcd0 2024-04-25T00:56:21.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.403+0000 7f90767fc640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f90640678c0 0x7f9064069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:21.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.403+0000 7f90767fc640 1 -- 172.21.15.112:0/604678447 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f9078121060 con 0x7f908014fcd0 2024-04-25T00:56:21.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.403+0000 7f9085f67640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f90640678c0 0x7f9064069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:21.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.403+0000 7f9085f67640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f90640678c0 0x7f9064069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f907c009580 tx=0x7f907c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:21.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.407+0000 7f90767fc640 1 -- 172.21.15.112:0/604678447 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90780c6030 con 0x7f908014fcd0 2024-04-25T00:56:21.586 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:21.583+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f908004f5f0 con 0x7f908014fcd0 2024-04-25T00:56:22.249 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.247+0000 7f90767fc640 1 -- 172.21.15.112:0/604678447 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f9078016600 con 0x7f908014fcd0 2024-04-25T00:56:22.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f90640678c0 msgr2=0x7f9064069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f90640678c0 0x7f9064069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f907c009580 tx=0x7f907c009190 comp rx=0 tx=0).stop 2024-04-25T00:56:22.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908014fcd0 msgr2=0x7f90801511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908014fcd0 0x7f90801511e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f907801a5b0 tx=0x7f9078019720 comp rx=0 tx=0).stop 2024-04-25T00:56:22.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 shutdown_connections 2024-04-25T00:56:22.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f908014fcd0 0x7f90801511e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f90800b9030 0x7f9080150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.253 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f90640678c0 0x7f9064069d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 --2- 172.21.15.112:0/604678447 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f908009b800 0x7f90801e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 >> 172.21.15.112:0/604678447 conn(0x7f908009eaf0 msgr2=0x7f908013fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:22.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 shutdown_connections 2024-04-25T00:56:22.254 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.251+0000 7f90881cb640 1 -- 172.21.15.112:0/604678447 wait complete. 2024-04-25T00:56:22.272 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T00:56:22.272 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:22.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.395+0000 7f079883b640 1 Processor -- start 2024-04-25T00:56:22.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.395+0000 7f079883b640 1 -- start start 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f079883b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07940745b0 0x7f07940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f079883b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f07940b3b40 0x7f07941487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f079883b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0794142370 0x7f0794142800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f079883b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f0794074990 con 0x7f0794142370 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f079883b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f079409c750 con 0x7f07940b3b40 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f079883b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f07940b9970 con 0x7f07940745b0 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0792d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0794142370 0x7f0794142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0792d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0794142370 0x7f0794142800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33758/0 (socket says 172.21.15.112:33758) 2024-04-25T00:56:22.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0792d9d640 1 -- 172.21.15.112:0/1484266530 learned_addr learned my addr 172.21.15.112:0/1484266530 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0791d9b640 1 --2- 172.21.15.112:0/1484266530 >> v2:172.21.15.187:3300/0 conn(0x7f07940b3b40 0x7f07941487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f079259c640 1 --2- 172.21.15.112:0/1484266530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07940745b0 0x7f07940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0792d9d640 1 -- 172.21.15.112:0/1484266530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07940745b0 msgr2=0x7f07940b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0792d9d640 1 --2- 172.21.15.112:0/1484266530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07940745b0 0x7f07940b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0792d9d640 1 -- 172.21.15.112:0/1484266530 >> v2:172.21.15.187:3300/0 conn(0x7f07940b3b40 msgr2=0x7f07941487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0792d9d640 1 --2- 172.21.15.112:0/1484266530 >> v2:172.21.15.187:3300/0 conn(0x7f07940b3b40 0x7f07941487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.399+0000 7f0792d9d640 1 -- 172.21.15.112:0/1484266530 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07940b9c50 con 0x7f0794142370 2024-04-25T00:56:22.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f0792d9d640 1 --2- 172.21.15.112:0/1484266530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0794142370 0x7f0794142800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f07940b03f0 tx=0x7f078c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dacc5041cc2b14e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:22.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079159a640 1 -- 172.21.15.112:0/1484266530 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0a5840 con 0x7f0794142370 2024-04-25T00:56:22.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079159a640 1 -- 172.21.15.112:0/1484266530 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078c0a59e0 con 0x7f0794142370 2024-04-25T00:56:22.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079159a640 1 -- 172.21.15.112:0/1484266530 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c0a5db0 con 0x7f0794142370 2024-04-25T00:56:22.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 -- 172.21.15.112:0/1484266530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0794142370 msgr2=0x7f0794142800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 --2- 172.21.15.112:0/1484266530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0794142370 0x7f0794142800 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f07940b03f0 tx=0x7f078c01c9c0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 -- 172.21.15.112:0/1484266530 shutdown_connections 2024-04-25T00:56:22.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 --2- 172.21.15.112:0/1484266530 >> v2:172.21.15.187:3300/0 conn(0x7f07940b3b40 0x7f07941487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 --2- 172.21.15.112:0/1484266530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0794142370 0x7f0794142800 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 --2- 172.21.15.112:0/1484266530 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f07940745b0 0x7f07940b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 -- 172.21.15.112:0/1484266530 >> 172.21.15.112:0/1484266530 conn(0x7f07940b6240 msgr2=0x7f07940b9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:22.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 -- 172.21.15.112:0/1484266530 shutdown_connections 2024-04-25T00:56:22.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 -- 172.21.15.112:0/1484266530 wait complete. 2024-04-25T00:56:22.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.403+0000 7f079883b640 1 Processor -- start 2024-04-25T00:56:22.406 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 -- start start 2024-04-25T00:56:22.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07940745b0 0x7f07941e09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07940b3b40 0x7f079413c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0794142370 0x7f07941d6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f079409b240 con 0x7f07940745b0 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0794142d40 con 0x7f07940b3b40 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f079409b3c0 con 0x7f0794142370 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07940745b0 0x7f07941e09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07940745b0 0x7f07941e09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33770/0 (socket says 172.21.15.112:33770) 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 -- 172.21.15.112:0/2293299360 learned_addr learned my addr 172.21.15.112:0/2293299360 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0791d9b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07940b3b40 0x7f079413c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0792d9d640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0794142370 0x7f07941d6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 -- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0794142370 msgr2=0x7f07941d6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0794142370 0x7f07941d6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 -- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07940b3b40 msgr2=0x7f079413c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07940b3b40 0x7f079413c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 -- 172.21.15.112:0/2293299360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07941e1620 con 0x7f07940745b0 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0791d9b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07940b3b40 0x7f079413c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079259c640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07940745b0 0x7f07941e09d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f078400e960 tx=0x7f078400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0782ffd640 1 -- 172.21.15.112:0/2293299360 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0784017070 con 0x7f07940745b0 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0782ffd640 1 -- 172.21.15.112:0/2293299360 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078400ca00 con 0x7f07940745b0 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07941e1340 con 0x7f07940745b0 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0782ffd640 1 -- 172.21.15.112:0/2293299360 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078400cdb0 con 0x7f07940745b0 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0794143900 con 0x7f07940745b0 2024-04-25T00:56:22.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f079404f580 con 0x7f07940745b0 2024-04-25T00:56:22.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0782ffd640 1 -- 172.21.15.112:0/2293299360 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f078401a070 con 0x7f07940745b0 2024-04-25T00:56:22.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0782ffd640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07740678f0 0x7f0774069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0782ffd640 1 -- 172.21.15.112:0/2293299360 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f0784085cb0 con 0x7f07940745b0 2024-04-25T00:56:22.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0791d9b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07740678f0 0x7f0774069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.407+0000 7f0791d9b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07740678f0 0x7f0774069db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0788009a60 tx=0x7f0788008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:22.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.411+0000 7f0782ffd640 1 -- 172.21.15.112:0/2293299360 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f07840567f0 con 0x7f07940745b0 2024-04-25T00:56:22.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.603+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f079406fb70 con 0x7f07940745b0 2024-04-25T00:56:22.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.603+0000 7f0782ffd640 1 -- 172.21.15.112:0/2293299360 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3537 (secure 0 0 0) 0x7f07840560f0 con 0x7f07940745b0 2024-04-25T00:56:22.606 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:22.606 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3898281312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3898281312},{"type":"v1","addr":"172.21.15.187:6833","nonce":3898281312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:56:18.588797+0000","modified":"2024-04-25T00:56:21.588160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3222796217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3222796217},{"type":"v1","addr":"172.21.15.112:6835","nonce":3222796217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-04-25T00:56:22.607 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07740678f0 msgr2=0x7f0774069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07740678f0 0x7f0774069db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0788009a60 tx=0x7f0788008040 comp rx=0 tx=0).stop 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07940745b0 msgr2=0x7f07941e09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07940745b0 0x7f07941e09d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f078400e960 tx=0x7f078400ee20 comp rx=0 tx=0).stop 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 shutdown_connections 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f0794142370 0x7f07941d6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f07740678f0 0x7f0774069db0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f07940b3b40 0x7f079413c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 --2- 172.21.15.112:0/2293299360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07940745b0 0x7f07941e09d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 >> 172.21.15.112:0/2293299360 conn(0x7f07940b6240 msgr2=0x7f079406cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 shutdown_connections 2024-04-25T00:56:22.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.607+0000 7f079883b640 1 -- 172.21.15.112:0/2293299360 wait complete. 2024-04-25T00:56:22.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.739+0000 7fd1a1808640 1 Processor -- start 2024-04-25T00:56:22.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.739+0000 7fd1a1808640 1 -- start start 2024-04-25T00:56:22.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd1a1808640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c0722d0 0x7fd19c072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd1a1808640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd19c147650 0x7fd19c151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd1a1808640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd19c0bbec0 con 0x7fd19c09c270 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd1a1808640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd19c0baaf0 con 0x7fd19c147650 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd1a1808640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c073100 con 0x7fd19c0722d0 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd19c147650 0x7fd19c151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd19c147650 0x7fd19c151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33780/0 (socket says 172.21.15.112:33780) 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19a7fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd19c09c270 0x7fd19c147d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:41330/0 (socket says 172.21.15.112:41330) 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19b7fe640 1 -- 172.21.15.112:0/3941020148 learned_addr learned my addr 172.21.15.112:0/3941020148 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19affd640 1 --2- 172.21.15.112:0/3941020148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c0722d0 0x7fd19c072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19b7fe640 1 -- 172.21.15.112:0/3941020148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c0722d0 msgr2=0x7fd19c072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19b7fe640 1 --2- 172.21.15.112:0/3941020148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c0722d0 0x7fd19c072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd199ffb640 1 -- 172.21.15.112:0/3941020148 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962011307 0 0) 0x7fd19c0bbec0 con 0x7fd19c09c270 2024-04-25T00:56:22.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19b7fe640 1 -- 172.21.15.112:0/3941020148 >> v1:172.21.15.112:6789/0 conn(0x7fd19c09c270 legacy=0x7fd19c147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19b7fe640 1 -- 172.21.15.112:0/3941020148 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c0bc5c0 con 0x7fd19c147650 2024-04-25T00:56:22.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd19b7fe640 1 --2- 172.21.15.112:0/3941020148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c147650 0x7fd19c151ef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd194009e50 tx=0x7fd19401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152e3f0ec9a33eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:22.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd199ffb640 1 -- 172.21.15.112:0/3941020148 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1940a58d0 con 0x7fd19c147650 2024-04-25T00:56:22.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd199ffb640 1 -- 172.21.15.112:0/3941020148 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1940a5a70 con 0x7fd19c147650 2024-04-25T00:56:22.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd199ffb640 1 -- 172.21.15.112:0/3941020148 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1940a5e40 con 0x7fd19c147650 2024-04-25T00:56:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd1a1808640 1 -- 172.21.15.112:0/3941020148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c147650 msgr2=0x7fd19c151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.743+0000 7fd1a1808640 1 --2- 172.21.15.112:0/3941020148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c147650 0x7fd19c151ef0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd194009e50 tx=0x7fd19401ca00 comp rx=0 tx=0).stop 2024-04-25T00:56:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- 172.21.15.112:0/3941020148 shutdown_connections 2024-04-25T00:56:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 --2- 172.21.15.112:0/3941020148 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c147650 0x7fd19c151ef0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 --2- 172.21.15.112:0/3941020148 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c0722d0 0x7fd19c072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- 172.21.15.112:0/3941020148 >> 172.21.15.112:0/3941020148 conn(0x7fd19c09e8c0 msgr2=0x7fd19c09ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- 172.21.15.112:0/3941020148 shutdown_connections 2024-04-25T00:56:22.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- 172.21.15.112:0/3941020148 wait complete. 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 Processor -- start 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- start start 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c0722d0 0x7fd19c14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c09c270 0x7fd19c0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd19c147650 0x7fd19c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c148410 con 0x7fd19c0722d0 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c09ac50 con 0x7fd19c147650 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd19c148590 con 0x7fd19c09c270 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c0722d0 0x7fd19c14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c0722d0 0x7fd19c14ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:33786/0 (socket says 172.21.15.112:33786) 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 -- 172.21.15.112:0/323420306 learned_addr learned my addr 172.21.15.112:0/323420306 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:22.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19a7fc640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c09c270 0x7fd19c0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19b7fe640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd19c147650 0x7fd19c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 -- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c09c270 msgr2=0x7fd19c0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c09c270 0x7fd19c0b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 -- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd19c147650 msgr2=0x7fd19c0b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd19c147650 0x7fd19c0b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 -- 172.21.15.112:0/323420306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c0b2d00 con 0x7fd19c0722d0 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19a7fc640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c09c270 0x7fd19c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd19affd640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c0722d0 0x7fd19c14ac30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d620 tx=0x7fd19000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd183fff640 1 -- 172.21.15.112:0/323420306 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19000b620 con 0x7fd19c0722d0 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd183fff640 1 -- 172.21.15.112:0/323420306 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd190019070 con 0x7fd19c0722d0 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd183fff640 1 -- 172.21.15.112:0/323420306 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19000b9e0 con 0x7fd19c0722d0 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c0b2fe0 con 0x7fd19c0722d0 2024-04-25T00:56:22.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c0b0e80 con 0x7fd19c0722d0 2024-04-25T00:56:22.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.747+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd19c04f580 con 0x7fd19c0722d0 2024-04-25T00:56:22.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.751+0000 7fd183fff640 1 -- 172.21.15.112:0/323420306 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd19000bb80 con 0x7fd19c0722d0 2024-04-25T00:56:22.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.751+0000 7fd183fff640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd1880678c0 0x7fd188069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:22.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.751+0000 7fd183fff640 1 -- 172.21.15.112:0/323420306 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fd190086060 con 0x7fd19c0722d0 2024-04-25T00:56:22.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.751+0000 7fd19a7fc640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd1880678c0 0x7fd188069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:22.753 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.751+0000 7fd19a7fc640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd1880678c0 0x7fd188069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd19c14b3e0 tx=0x7fd18c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:22.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.755+0000 7fd183fff640 1 -- 172.21.15.112:0/323420306 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd190056c10 con 0x7fd19c0722d0 2024-04-25T00:56:22.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.903+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd19c1473f0 con 0x7fd19c0722d0 2024-04-25T00:56:22.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.903+0000 7fd183fff640 1 -- 172.21.15.112:0/323420306 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16717 (secure 0 0 0) 0x7fd190056510 con 0x7fd19c0722d0 2024-04-25T00:56:22.907 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:22.907 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:20.226650+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:56:16.783920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:56:17.569481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd1880678c0 msgr2=0x7fd188069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd1880678c0 0x7fd188069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd19c14b3e0 tx=0x7fd18c009040 comp rx=0 tx=0).stop 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c0722d0 msgr2=0x7fd19c14ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c0722d0 0x7fd19c14ac30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd19000d620 tx=0x7fd19000dae0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 shutdown_connections 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd1880678c0 0x7fd188069d80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd19c147650 0x7fd19c0b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd19c09c270 0x7fd19c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 --2- 172.21.15.112:0/323420306 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd19c0722d0 0x7fd19c14ac30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 >> 172.21.15.112:0/323420306 conn(0x7fd19c09e8c0 msgr2=0x7fd19c09e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 shutdown_connections 2024-04-25T00:56:22.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:22.907+0000 7fd1a1808640 1 -- 172.21.15.112:0/323420306 wait complete. 2024-04-25T00:56:22.925 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:56:22.925 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:56:23.024 INFO:tasks.ceph.mds.a.smithi112.stderr:2024-04-25T00:56:23.023+0000 7faff1480640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:56:23.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f295931c640 1 Processor -- start 2024-04-25T00:56:23.039 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f295931c640 1 -- start start 2024-04-25T00:56:23.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f295931c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29540b7800 0x7f29540b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f295931c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b5510 0x7f2954145150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f295931c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29540b7be0 con 0x7f29540b4320 2024-04-25T00:56:23.040 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f295931c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f2954145690 con 0x7f29540b7800 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f295931c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29540b5c50 con 0x7f29540b5510 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f2952ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29540b7800 0x7f29540b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f2952ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29540b7800 0x7f29540b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57704/0 (socket says 172.21.15.112:57704) 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f2952ffd640 1 -- 172.21.15.112:0/3916616521 learned_addr learned my addr 172.21.15.112:0/3916616521 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f29527fc640 1 --2- 172.21.15.112:0/3916616521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b5510 0x7f2954145150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f2952ffd640 1 -- 172.21.15.112:0/3916616521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b5510 msgr2=0x7f2954145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f2952ffd640 1 --2- 172.21.15.112:0/3916616521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b5510 0x7f2954145150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f2952ffd640 1 -- 172.21.15.112:0/3916616521 >> v1:172.21.15.187:6789/0 conn(0x7f29540b4320 legacy=0x7f2954148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f2952ffd640 1 -- 172.21.15.112:0/3916616521 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29540b6370 con 0x7f29540b7800 2024-04-25T00:56:23.041 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.039+0000 7f29527fc640 1 --2- 172.21.15.112:0/3916616521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b5510 0x7f2954145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:23.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f2952ffd640 1 --2- 172.21.15.112:0/3916616521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b7800 0x7f29540b6b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2944009870 tx=0x7f294401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9a145132f0a68f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.042 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f2951ffb640 1 -- 172.21.15.112:0/3916616521 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29440b0070 con 0x7f29540b7800 2024-04-25T00:56:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f2951ffb640 1 -- 172.21.15.112:0/3916616521 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29440ab600 con 0x7f29540b7800 2024-04-25T00:56:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f2951ffb640 1 -- 172.21.15.112:0/3916616521 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29440ab9d0 con 0x7f29540b7800 2024-04-25T00:56:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- 172.21.15.112:0/3916616521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b7800 msgr2=0x7f29540b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.043 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 --2- 172.21.15.112:0/3916616521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b7800 0x7f29540b6b70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2944009870 tx=0x7f294401c490 comp rx=0 tx=0).stop 2024-04-25T00:56:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- 172.21.15.112:0/3916616521 shutdown_connections 2024-04-25T00:56:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 --2- 172.21.15.112:0/3916616521 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b5510 0x7f2954145150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 --2- 172.21.15.112:0/3916616521 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b7800 0x7f29540b6b70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- 172.21.15.112:0/3916616521 >> 172.21.15.112:0/3916616521 conn(0x7f2954066f50 msgr2=0x7f2954067360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- 172.21.15.112:0/3916616521 shutdown_connections 2024-04-25T00:56:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- 172.21.15.112:0/3916616521 wait complete. 2024-04-25T00:56:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 Processor -- start 2024-04-25T00:56:23.044 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- start start 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b4320 0x7f2954142360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b5510 0x7f2954140a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29540b7800 0x7f2954141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f2954152af0 con 0x7f29540b5510 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29540b2a50 con 0x7f29540b7800 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f2954152c70 con 0x7f29540b4320 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b5510 0x7f2954140a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b5510 0x7f2954140a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57708/0 (socket says 172.21.15.112:57708) 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 -- 172.21.15.112:0/2046415728 learned_addr learned my addr 172.21.15.112:0/2046415728 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29537fe640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29540b7800 0x7f2954141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f2952ffd640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b4320 0x7f2954142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 -- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b4320 msgr2=0x7f2954142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b4320 0x7f2954142360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.047 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 -- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29540b7800 msgr2=0x7f2954141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29540b7800 0x7f2954141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 -- 172.21.15.112:0/2046415728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2954140f60 con 0x7f29540b5510 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29537fe640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29540b7800 0x7f2954141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f29527fc640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b5510 0x7f2954140a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f294800c7f0 tx=0x7f294800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f293b7fe640 1 -- 172.21.15.112:0/2046415728 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948007a30 con 0x7f29540b5510 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f293b7fe640 1 -- 172.21.15.112:0/2046415728 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2948007bd0 con 0x7f29540b5510 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f293b7fe640 1 -- 172.21.15.112:0/2046415728 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29480113f0 con 0x7f29540b5510 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29540b8a20 con 0x7f29540b5510 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.043+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2954140220 con 0x7f29540b5510 2024-04-25T00:56:23.048 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.047+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f295404f580 con 0x7f29540b5510 2024-04-25T00:56:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.047+0000 7f293b7fe640 1 -- 172.21.15.112:0/2046415728 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2948011590 con 0x7f29540b5510 2024-04-25T00:56:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.047+0000 7f293b7fe640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29400678c0 0x7f2940069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.047+0000 7f293b7fe640 1 -- 172.21.15.112:0/2046415728 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f29480861d0 con 0x7f29540b5510 2024-04-25T00:56:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.047+0000 7f2952ffd640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29400678c0 0x7f2940069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.047+0000 7f2952ffd640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29400678c0 0x7f2940069d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29440ad000 tx=0x7f2944008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.052 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.051+0000 7f293b7fe640 1 -- 172.21.15.112:0/2046415728 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2948015020 con 0x7f29540b5510 2024-04-25T00:56:23.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.207+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29540b3f60 con 0x7f29400678c0 2024-04-25T00:56:23.225 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.223+0000 7f293b7fe640 1 -- 172.21.15.112:0/2046415728 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239851 (secure 0 0 0) 0x7f29540b3f60 con 0x7f29400678c0 2024-04-25T00:56:23.228 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:23.229 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:56:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29400678c0 msgr2=0x7f2940069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.230 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29400678c0 0x7f2940069d80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f29440ad000 tx=0x7f2944008040 comp rx=0 tx=0).stop 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b5510 msgr2=0x7f2954140a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b5510 0x7f2954140a20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f294800c7f0 tx=0x7f294800ccb0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 shutdown_connections 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29400678c0 0x7f2940069d80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29540b7800 0x7f2954141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29540b5510 0x7f2954140a20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 --2- 172.21.15.112:0/2046415728 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29540b4320 0x7f2954142360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 >> 172.21.15.112:0/2046415728 conn(0x7f2954066f50 msgr2=0x7f2954144810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 shutdown_connections 2024-04-25T00:56:23.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.227+0000 7f295931c640 1 -- 172.21.15.112:0/2046415728 wait complete. 2024-04-25T00:56:23.244 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-04-25T00:56:21.526227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219500,"kb_used_data":3316,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755220,"statfs":{"total":773094113280,"available":772869345280,"internally_reserved":0,"allocated":3395584,"data_stored":1977377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.302547"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.233385+0000","last_change":"2024-04-25T00:56:17.233385+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.233385+0000","last_undegraded":"2024-04-25T00:56:17.233385+0000","last_fullsized":"2024-04-25T00:56:17.233385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.258720+0000","last_change":"2024-04-25T00:56:17.258720+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.258720+0000","last_undegraded":"2024-04-25T00:56:17.258720+0000","last_fullsized":"2024-04-25T00:56:17.258720+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.231789+0000","last_change":"2024-04-25T00:56:17.231789+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.231789+0000","last_undegraded":"2024-04-25T00:56:17.231789+0000","last_fullsized":"2024-04-25T00:56:17.231789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.230806+0000","last_change":"2024-04-25T00:56:17.230806+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.230806+0000","last_undegraded":"2024-04-25T00:56:17.230806+0000","last_fullsized":"2024-04-25T00:56:17.230806+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.227171+0000","last_change":"2024-04-25T00:56:17.227171+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.227171+0000","last_undegraded":"2024-04-25T00:56:17.227171+0000","last_fullsized":"2024-04-25T00:56:17.227171+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.225331+0000","last_change":"2024-04-25T00:56:17.225331+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.225331+0000","last_undegraded":"2024-04-25T00:56:17.225331+0000","last_fullsized":"2024-04-25T00:56:17.225331+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.230582+0000","last_change":"2024-04-25T00:56:17.230582+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.230582+0000","last_undegraded":"2024-04-25T00:56:17.230582+0000","last_fullsized":"2024-04-25T00:56:17.230582+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.225108+0000","last_change":"2024-04-25T00:56:17.225108+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.225108+0000","last_undegraded":"2024-04-25T00:56:17.225108+0000","last_fullsized":"2024-04-25T00:56:17.225108+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.231067+0000","last_change":"2024-04-25T00:56:17.231067+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.231067+0000","last_undegraded":"2024-04-25T00:56:17.231067+0000","last_fullsized":"2024-04-25T00:56:17.231067+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.258249+0000","last_change":"2024-04-25T00:56:17.258249+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.258249+0000","last_undegraded":"2024-04-25T00:56:17.258249+0000","last_fullsized":"2024-04-25T00:56:17.258249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.224944+0000","last_change":"2024-04-25T00:56:17.224944+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.224944+0000","last_undegraded":"2024-04-25T00:56:17.224944+0000","last_fullsized":"2024-04-25T00:56:17.224944+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.229509+0000","last_change":"2024-04-25T00:56:17.229509+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.229509+0000","last_undegraded":"2024-04-25T00:56:17.229509+0000","last_fullsized":"2024-04-25T00:56:17.229509+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.226114+0000","last_change":"2024-04-25T00:56:17.226114+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.226114+0000","last_undegraded":"2024-04-25T00:56:17.226114+0000","last_fullsized":"2024-04-25T00:56:17.226114+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.234254+0000","last_change":"2024-04-25T00:56:17.234254+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.234254+0000","last_undegraded":"2024-04-25T00:56:17.234254+0000","last_fullsized":"2024-04-25T00:56:17.234254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.229865+0000","last_change":"2024-04-25T00:56:17.229865+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.229865+0000","last_undegraded":"2024-04-25T00:56:17.229865+0000","last_fullsized":"2024-04-25T00:56:17.229865+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.229231+0000","last_change":"2024-04-25T00:56:17.229231+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.229231+0000","last_undegraded":"2024-04-25T00:56:17.229231+0000","last_fullsized":"2024-04-25T00:56:17.229231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.234832+0000","last_change":"2024-04-25T00:56:17.234832+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.234832+0000","last_undegraded":"2024-04-25T00:56:17.234832+0000","last_fullsized":"2024-04-25T00:56:17.234832+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.227086+0000","last_change":"2024-04-25T00:56:17.227086+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.227086+0000","last_undegraded":"2024-04-25T00:56:17.227086+0000","last_fullsized":"2024-04-25T00:56:17.227086+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.227190+0000","last_change":"2024-04-25T00:56:17.227190+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.227190+0000","last_undegraded":"2024-04-25T00:56:17.227190+0000","last_fullsized":"2024-04-25T00:56:17.227190+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.226504+0000","last_change":"2024-04-25T00:56:17.226504+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.226504+0000","last_undegraded":"2024-04-25T00:56:17.226504+0000","last_fullsized":"2024-04-25T00:56:17.226504+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.258088+0000","last_change":"2024-04-25T00:56:17.258088+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.258088+0000","last_undegraded":"2024-04-25T00:56:17.258088+0000","last_fullsized":"2024-04-25T00:56:17.258088+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.233245+0000","last_change":"2024-04-25T00:56:17.233245+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.233245+0000","last_undegraded":"2024-04-25T00:56:17.233245+0000","last_fullsized":"2024-04-25T00:56:17.233245+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.229093+0000","last_change":"2024-04-25T00:56:17.229093+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.229093+0000","last_undegraded":"2024-04-25T00:56:17.229093+0000","last_fullsized":"2024-04-25T00:56:17.229093+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.224926+0000","last_change":"2024-04-25T00:56:17.224926+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.224926+0000","last_undegraded":"2024-04-25T00:56:17.224926+0000","last_fullsized":"2024-04-25T00:56:17.224926+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.232697+0000","last_change":"2024-04-25T00:56:17.232697+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.232697+0000","last_undegraded":"2024-04-25T00:56:17.232697+0000","last_fullsized":"2024-04-25T00:56:17.232697+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.257496+0000","last_change":"2024-04-25T00:56:17.257496+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.257496+0000","last_undegraded":"2024-04-25T00:56:17.257496+0000","last_fullsized":"2024-04-25T00:56:17.257496+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.232588+0000","last_change":"2024-04-25T00:56:17.232588+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.232588+0000","last_undegraded":"2024-04-25T00:56:17.232588+0000","last_fullsized":"2024-04-25T00:56:17.232588+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.257421+0000","last_change":"2024-04-25T00:56:17.257421+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.257421+0000","last_undegraded":"2024-04-25T00:56:17.257421+0000","last_fullsized":"2024-04-25T00:56:17.257421+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.233147+0000","last_change":"2024-04-25T00:56:17.233147+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.233147+0000","last_undegraded":"2024-04-25T00:56:17.233147+0000","last_fullsized":"2024-04-25T00:56:17.233147+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.228181+0000","last_change":"2024-04-25T00:56:17.228181+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.228181+0000","last_undegraded":"2024-04-25T00:56:17.228181+0000","last_fullsized":"2024-04-25T00:56:17.228181+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.228550+0000","last_change":"2024-04-25T00:56:17.228550+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.228550+0000","last_undegraded":"2024-04-25T00:56:17.228550+0000","last_fullsized":"2024-04-25T00:56:17.228550+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T00:56:18.221586+0000","last_change":"2024-04-25T00:56:18.221586+0000","last_active":"2024-04-25T00:56:18.221586+0000","last_peered":"2024-04-25T00:56:18.221586+0000","last_clean":"2024-04-25T00:56:18.221586+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:18.221586+0000","last_undegraded":"2024-04-25T00:56:18.221586+0000","last_fullsized":"2024-04-25T00:56:18.221586+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":26,"state":"creating+peering","last_fresh":"2024-04-25T00:56:17.234454+0000","last_change":"2024-04-25T00:56:17.234454+0000","last_active":"2024-04-25T00:56:17.216633+0000","last_peered":"2024-04-25T00:56:17.216633+0000","last_clean":"2024-04-25T00:56:17.216633+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T00:56:17.234454+0000","last_undegraded":"2024-04-25T00:56:17.234454+0000","last_fullsized":"2024-04-25T00:56:17.234454+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-04-25T00:56:17.231191+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:56:17.231191+0000","last_peered":"2024-04-25T00:56:17.231191+0000","last_clean":"2024-04-25T00:56:17.231191+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:56:17.231191+0000","last_undegraded":"2024-04-25T00:56:17.231191+0000","last_fullsized":"2024-04-25T00:56:17.231191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":689221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":98853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":104670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":98853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":98853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":98853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":98853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":689221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:56:23.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:56:23.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.407+0000 7f715a056640 1 Processor -- start 2024-04-25T00:56:23.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.407+0000 7f715a056640 1 -- start start 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f7154072400 0x7f71540b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71540b7030 0x7f71540b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71540727e0 con 0x7f7154148820 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f715413fa70 con 0x7f71540b7030 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f715406ed30 con 0x7f7154072400 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f7152ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71540b7030 0x7f71540b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f7152ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71540b7030 0x7f71540b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57710/0 (socket says 172.21.15.112:57710) 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f7152ffd640 1 -- 172.21.15.112:0/3734532858 learned_addr learned my addr 172.21.15.112:0/3734532858 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f71537fe640 1 --2- 172.21.15.112:0/3734532858 >> v2:172.21.15.187:3300/0 conn(0x7f7154072400 0x7f71540b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f7152ffd640 1 -- 172.21.15.112:0/3734532858 >> v2:172.21.15.187:3300/0 conn(0x7f7154072400 msgr2=0x7f71540b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:23.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f7152ffd640 1 --2- 172.21.15.112:0/3734532858 >> v2:172.21.15.187:3300/0 conn(0x7f7154072400 0x7f71540b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f7152ffd640 1 -- 172.21.15.112:0/3734532858 >> v1:172.21.15.187:6789/0 conn(0x7f7154148820 legacy=0x7f71540b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f7152ffd640 1 -- 172.21.15.112:0/3734532858 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71540b29a0 con 0x7f71540b7030 2024-04-25T00:56:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f71537fe640 1 --2- 172.21.15.112:0/3734532858 >> v2:172.21.15.187:3300/0 conn(0x7f7154072400 0x7f71540b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:23.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f7152ffd640 1 --2- 172.21.15.112:0/3734532858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71540b7030 0x7f71540b4870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7154072c00 tx=0x7f714801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944600efaeb9375d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f71527fc640 1 -- 172.21.15.112:0/3734532858 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71480b0070 con 0x7f71540b7030 2024-04-25T00:56:23.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f71527fc640 1 -- 172.21.15.112:0/3734532858 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71480ab600 con 0x7f71540b7030 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f71527fc640 1 -- 172.21.15.112:0/3734532858 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71480ab9d0 con 0x7f71540b7030 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 -- 172.21.15.112:0/3734532858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71540b7030 msgr2=0x7f71540b4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 --2- 172.21.15.112:0/3734532858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71540b7030 0x7f71540b4870 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7154072c00 tx=0x7f714801c450 comp rx=0 tx=0).stop 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 -- 172.21.15.112:0/3734532858 shutdown_connections 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 --2- 172.21.15.112:0/3734532858 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71540b7030 0x7f71540b4870 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 --2- 172.21.15.112:0/3734532858 >> v2:172.21.15.187:3300/0 conn(0x7f7154072400 0x7f71540b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.411+0000 7f715a056640 1 -- 172.21.15.112:0/3734532858 >> 172.21.15.112:0/3734532858 conn(0x7f71540b77a0 msgr2=0x7f71540b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- 172.21.15.112:0/3734532858 shutdown_connections 2024-04-25T00:56:23.415 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- 172.21.15.112:0/3734532858 wait complete. 2024-04-25T00:56:23.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 Processor -- start 2024-04-25T00:56:23.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- start start 2024-04-25T00:56:23.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7154072400 0x7f715413a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71540b7030 0x7f715413d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154148820 0x7f715413dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.416 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f715406e7f0 con 0x7f7154148820 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f715406e670 con 0x7f71540b7030 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7154152b70 con 0x7f7154072400 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154148820 0x7f715413dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154148820 0x7f715413dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57722/0 (socket says 172.21.15.112:57722) 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 -- 172.21.15.112:0/2373763423 learned_addr learned my addr 172.21.15.112:0/2373763423 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 -- 172.21.15.112:0/2373763423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7154072400 msgr2=0x7f715413a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7152ffd640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71540b7030 0x7f715413d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f71537fe640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7154072400 0x7f715413a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7154072400 0x7f715413a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 -- 172.21.15.112:0/2373763423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71540b7030 msgr2=0x7f715413d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71540b7030 0x7f715413d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.417 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 -- 172.21.15.112:0/2373763423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715413b9e0 con 0x7f7154148820 2024-04-25T00:56:23.418 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f7153fff640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154148820 0x7f715413dd80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f714c00b9c0 tx=0x7f714c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f713bfff640 1 -- 172.21.15.112:0/2373763423 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c019070 con 0x7f7154148820 2024-04-25T00:56:23.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f713bfff640 1 -- 172.21.15.112:0/2373763423 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714c00cb40 con 0x7f7154148820 2024-04-25T00:56:23.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715413bc50 con 0x7f7154148820 2024-04-25T00:56:23.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f713bfff640 1 -- 172.21.15.112:0/2373763423 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c014440 con 0x7f7154148820 2024-04-25T00:56:23.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715413ec30 con 0x7f7154148820 2024-04-25T00:56:23.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.415+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715404f580 con 0x7f7154148820 2024-04-25T00:56:23.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.419+0000 7f713bfff640 1 -- 172.21.15.112:0/2373763423 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f714c01c070 con 0x7f7154148820 2024-04-25T00:56:23.419 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.419+0000 7f713bfff640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f71400678c0 0x7f7140069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.419+0000 7f713bfff640 1 -- 172.21.15.112:0/2373763423 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f714c085990 con 0x7f7154148820 2024-04-25T00:56:23.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.419+0000 7f71537fe640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f71400678c0 0x7f7140069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.420 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.419+0000 7f71537fe640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f71400678c0 0x7f7140069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7144006fd0 tx=0x7f7144008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.423+0000 7f713bfff640 1 -- 172.21.15.112:0/2373763423 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f714c056560 con 0x7f7154148820 2024-04-25T00:56:23.569 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.567+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71540b91a0 con 0x7f71400678c0 2024-04-25T00:56:23.577 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.575+0000 7f713bfff640 1 -- 172.21.15.112:0/2373763423 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f71540b91a0 con 0x7f71400678c0 2024-04-25T00:56:23.579 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:23.582 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f71400678c0 msgr2=0x7f7140069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f71400678c0 0x7f7140069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7144006fd0 tx=0x7f7144008040 comp rx=0 tx=0).stop 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154148820 msgr2=0x7f715413dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154148820 0x7f715413dd80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f714c00b9c0 tx=0x7f714c00be80 comp rx=0 tx=0).stop 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 shutdown_connections 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7154148820 0x7f715413dd80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f71400678c0 0x7f7140069d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f71540b7030 0x7f715413d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 --2- 172.21.15.112:0/2373763423 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7154072400 0x7f715413a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 >> 172.21.15.112:0/2373763423 conn(0x7f71540b77a0 msgr2=0x7f71540b96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:23.583 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 shutdown_connections 2024-04-25T00:56:23.584 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.579+0000 7f715a056640 1 -- 172.21.15.112:0/2373763423 wait complete. 2024-04-25T00:56:23.598 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-04-25T00:56:23.526746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224104,"kb_used_data":3752,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750616,"statfs":{"total":773094113280,"available":772864630784,"internally_reserved":0,"allocated":3842048,"data_stored":2177652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001663"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.027208+0000","last_change":"2024-04-25T00:56:19.283843+0000","last_active":"2024-04-25T00:56:21.027208+0000","last_peered":"2024-04-25T00:56:21.027208+0000","last_clean":"2024-04-25T00:56:21.027208+0000","last_became_active":"2024-04-25T00:56:19.279618+0000","last_became_peered":"2024-04-25T00:56:19.279618+0000","last_unstale":"2024-04-25T00:56:21.027208+0000","last_undegraded":"2024-04-25T00:56:21.027208+0000","last_fullsized":"2024-04-25T00:56:21.027208+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:45.897182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026729+0000","last_change":"2024-04-25T00:56:19.273688+0000","last_active":"2024-04-25T00:56:21.026729+0000","last_peered":"2024-04-25T00:56:21.026729+0000","last_clean":"2024-04-25T00:56:21.026729+0000","last_became_active":"2024-04-25T00:56:19.272097+0000","last_became_peered":"2024-04-25T00:56:19.272097+0000","last_unstale":"2024-04-25T00:56:21.026729+0000","last_undegraded":"2024-04-25T00:56:21.026729+0000","last_fullsized":"2024-04-25T00:56:21.026729+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:56:11.819933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.237654+0000","last_change":"2024-04-25T00:56:19.268436+0000","last_active":"2024-04-25T00:56:20.237654+0000","last_peered":"2024-04-25T00:56:20.237654+0000","last_clean":"2024-04-25T00:56:20.237654+0000","last_became_active":"2024-04-25T00:56:19.267677+0000","last_became_peered":"2024-04-25T00:56:19.267677+0000","last_unstale":"2024-04-25T00:56:20.237654+0000","last_undegraded":"2024-04-25T00:56:20.237654+0000","last_fullsized":"2024-04-25T00:56:20.237654+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:19.210139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275664+0000","last_change":"2024-04-25T00:56:19.267884+0000","last_active":"2024-04-25T00:56:20.275664+0000","last_peered":"2024-04-25T00:56:20.275664+0000","last_clean":"2024-04-25T00:56:20.275664+0000","last_became_active":"2024-04-25T00:56:19.267527+0000","last_became_peered":"2024-04-25T00:56:19.267527+0000","last_unstale":"2024-04-25T00:56:20.275664+0000","last_undegraded":"2024-04-25T00:56:20.275664+0000","last_fullsized":"2024-04-25T00:56:20.275664+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:00.230340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236223+0000","last_change":"2024-04-25T00:56:19.272071+0000","last_active":"2024-04-25T00:56:20.236223+0000","last_peered":"2024-04-25T00:56:20.236223+0000","last_clean":"2024-04-25T00:56:20.236223+0000","last_became_active":"2024-04-25T00:56:19.271764+0000","last_became_peered":"2024-04-25T00:56:19.271764+0000","last_unstale":"2024-04-25T00:56:20.236223+0000","last_undegraded":"2024-04-25T00:56:20.236223+0000","last_fullsized":"2024-04-25T00:56:20.236223+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:20.794354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025960+0000","last_change":"2024-04-25T00:56:19.271430+0000","last_active":"2024-04-25T00:56:21.025960+0000","last_peered":"2024-04-25T00:56:21.025960+0000","last_clean":"2024-04-25T00:56:21.025960+0000","last_became_active":"2024-04-25T00:56:19.269714+0000","last_became_peered":"2024-04-25T00:56:19.269714+0000","last_unstale":"2024-04-25T00:56:21.025960+0000","last_undegraded":"2024-04-25T00:56:21.025960+0000","last_fullsized":"2024-04-25T00:56:21.025960+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:46.918918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025242+0000","last_change":"2024-04-25T00:56:19.290843+0000","last_active":"2024-04-25T00:56:21.025242+0000","last_peered":"2024-04-25T00:56:21.025242+0000","last_clean":"2024-04-25T00:56:21.025242+0000","last_became_active":"2024-04-25T00:56:19.290400+0000","last_became_peered":"2024-04-25T00:56:19.290400+0000","last_unstale":"2024-04-25T00:56:21.025242+0000","last_undegraded":"2024-04-25T00:56:21.025242+0000","last_fullsized":"2024-04-25T00:56:21.025242+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:08.938239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.024588+0000","last_change":"2024-04-25T00:56:19.263059+0000","last_active":"2024-04-25T00:56:21.024588+0000","last_peered":"2024-04-25T00:56:21.024588+0000","last_clean":"2024-04-25T00:56:21.024588+0000","last_became_active":"2024-04-25T00:56:19.262139+0000","last_became_peered":"2024-04-25T00:56:19.262139+0000","last_unstale":"2024-04-25T00:56:21.024588+0000","last_undegraded":"2024-04-25T00:56:21.024588+0000","last_fullsized":"2024-04-25T00:56:21.024588+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:08.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.239119+0000","last_change":"2024-04-25T00:56:19.267367+0000","last_active":"2024-04-25T00:56:20.239119+0000","last_peered":"2024-04-25T00:56:20.239119+0000","last_clean":"2024-04-25T00:56:20.239119+0000","last_became_active":"2024-04-25T00:56:19.266630+0000","last_became_peered":"2024-04-25T00:56:19.266630+0000","last_unstale":"2024-04-25T00:56:20.239119+0000","last_undegraded":"2024-04-25T00:56:20.239119+0000","last_fullsized":"2024-04-25T00:56:20.239119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:08.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275572+0000","last_change":"2024-04-25T00:56:19.275053+0000","last_active":"2024-04-25T00:56:20.275572+0000","last_peered":"2024-04-25T00:56:20.275572+0000","last_clean":"2024-04-25T00:56:20.275572+0000","last_became_active":"2024-04-25T00:56:19.273993+0000","last_became_peered":"2024-04-25T00:56:19.273993+0000","last_unstale":"2024-04-25T00:56:20.275572+0000","last_undegraded":"2024-04-25T00:56:20.275572+0000","last_fullsized":"2024-04-25T00:56:20.275572+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:08.695801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025998+0000","last_change":"2024-04-25T00:56:19.269917+0000","last_active":"2024-04-25T00:56:21.025998+0000","last_peered":"2024-04-25T00:56:21.025998+0000","last_clean":"2024-04-25T00:56:21.025998+0000","last_became_active":"2024-04-25T00:56:19.269207+0000","last_became_peered":"2024-04-25T00:56:19.269207+0000","last_unstale":"2024-04-25T00:56:21.025998+0000","last_undegraded":"2024-04-25T00:56:21.025998+0000","last_fullsized":"2024-04-25T00:56:21.025998+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:46.918918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.235939+0000","last_change":"2024-04-25T00:56:19.272437+0000","last_active":"2024-04-25T00:56:20.235939+0000","last_peered":"2024-04-25T00:56:20.235939+0000","last_clean":"2024-04-25T00:56:20.235939+0000","last_became_active":"2024-04-25T00:56:19.271591+0000","last_became_peered":"2024-04-25T00:56:19.271591+0000","last_unstale":"2024-04-25T00:56:20.235939+0000","last_undegraded":"2024-04-25T00:56:20.235939+0000","last_fullsized":"2024-04-25T00:56:20.235939+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:44.346150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236562+0000","last_change":"2024-04-25T00:56:19.267746+0000","last_active":"2024-04-25T00:56:20.236562+0000","last_peered":"2024-04-25T00:56:20.236562+0000","last_clean":"2024-04-25T00:56:20.236562+0000","last_became_active":"2024-04-25T00:56:19.267418+0000","last_became_peered":"2024-04-25T00:56:19.267418+0000","last_unstale":"2024-04-25T00:56:20.236562+0000","last_undegraded":"2024-04-25T00:56:20.236562+0000","last_fullsized":"2024-04-25T00:56:20.236562+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:39.003174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026012+0000","last_change":"2024-04-25T00:56:19.271357+0000","last_active":"2024-04-25T00:56:21.026012+0000","last_peered":"2024-04-25T00:56:21.026012+0000","last_clean":"2024-04-25T00:56:21.026012+0000","last_became_active":"2024-04-25T00:56:19.270128+0000","last_became_peered":"2024-04-25T00:56:19.270128+0000","last_unstale":"2024-04-25T00:56:21.026012+0000","last_undegraded":"2024-04-25T00:56:21.026012+0000","last_fullsized":"2024-04-25T00:56:21.026012+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:19.210139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.237246+0000","last_change":"2024-04-25T00:56:19.290328+0000","last_active":"2024-04-25T00:56:20.237246+0000","last_peered":"2024-04-25T00:56:20.237246+0000","last_clean":"2024-04-25T00:56:20.237246+0000","last_became_active":"2024-04-25T00:56:19.289589+0000","last_became_peered":"2024-04-25T00:56:19.289589+0000","last_unstale":"2024-04-25T00:56:20.237246+0000","last_undegraded":"2024-04-25T00:56:20.237246+0000","last_fullsized":"2024-04-25T00:56:20.237246+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:46.918918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025333+0000","last_change":"2024-04-25T00:56:19.272787+0000","last_active":"2024-04-25T00:56:21.025333+0000","last_peered":"2024-04-25T00:56:21.025333+0000","last_clean":"2024-04-25T00:56:21.025333+0000","last_became_active":"2024-04-25T00:56:19.271626+0000","last_became_peered":"2024-04-25T00:56:19.271626+0000","last_unstale":"2024-04-25T00:56:21.025333+0000","last_undegraded":"2024-04-25T00:56:21.025333+0000","last_fullsized":"2024-04-25T00:56:21.025333+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:44.346150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026091+0000","last_change":"2024-04-25T00:56:19.283763+0000","last_active":"2024-04-25T00:56:21.026091+0000","last_peered":"2024-04-25T00:56:21.026091+0000","last_clean":"2024-04-25T00:56:21.026091+0000","last_became_active":"2024-04-25T00:56:19.282983+0000","last_became_peered":"2024-04-25T00:56:19.282983+0000","last_unstale":"2024-04-25T00:56:21.026091+0000","last_undegraded":"2024-04-25T00:56:21.026091+0000","last_fullsized":"2024-04-25T00:56:21.026091+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:19.210139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026898+0000","last_change":"2024-04-25T00:56:19.272364+0000","last_active":"2024-04-25T00:56:21.026898+0000","last_peered":"2024-04-25T00:56:21.026898+0000","last_clean":"2024-04-25T00:56:21.026898+0000","last_became_active":"2024-04-25T00:56:19.271806+0000","last_became_peered":"2024-04-25T00:56:19.271806+0000","last_unstale":"2024-04-25T00:56:21.026898+0000","last_undegraded":"2024-04-25T00:56:21.026898+0000","last_fullsized":"2024-04-25T00:56:21.026898+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:27:14.079356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026122+0000","last_change":"2024-04-25T00:56:19.269064+0000","last_active":"2024-04-25T00:56:21.026122+0000","last_peered":"2024-04-25T00:56:21.026122+0000","last_clean":"2024-04-25T00:56:21.026122+0000","last_became_active":"2024-04-25T00:56:19.268775+0000","last_became_peered":"2024-04-25T00:56:19.268775+0000","last_unstale":"2024-04-25T00:56:21.026122+0000","last_undegraded":"2024-04-25T00:56:21.026122+0000","last_fullsized":"2024-04-25T00:56:21.026122+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:09.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236505+0000","last_change":"2024-04-25T00:56:19.253117+0000","last_active":"2024-04-25T00:56:20.236505+0000","last_peered":"2024-04-25T00:56:20.236505+0000","last_clean":"2024-04-25T00:56:20.236505+0000","last_became_active":"2024-04-25T00:56:19.252391+0000","last_became_peered":"2024-04-25T00:56:19.252391+0000","last_unstale":"2024-04-25T00:56:20.236505+0000","last_undegraded":"2024-04-25T00:56:20.236505+0000","last_fullsized":"2024-04-25T00:56:20.236505+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:33.791550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.234977+0000","last_change":"2024-04-25T00:56:19.248971+0000","last_active":"2024-04-25T00:56:20.234977+0000","last_peered":"2024-04-25T00:56:20.234977+0000","last_clean":"2024-04-25T00:56:20.234977+0000","last_became_active":"2024-04-25T00:56:19.248190+0000","last_became_peered":"2024-04-25T00:56:19.248190+0000","last_unstale":"2024-04-25T00:56:20.234977+0000","last_undegraded":"2024-04-25T00:56:20.234977+0000","last_fullsized":"2024-04-25T00:56:20.234977+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:41.445476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236465+0000","last_change":"2024-04-25T00:56:19.271782+0000","last_active":"2024-04-25T00:56:20.236465+0000","last_peered":"2024-04-25T00:56:20.236465+0000","last_clean":"2024-04-25T00:56:20.236465+0000","last_became_active":"2024-04-25T00:56:19.271042+0000","last_became_peered":"2024-04-25T00:56:19.271042+0000","last_unstale":"2024-04-25T00:56:20.236465+0000","last_undegraded":"2024-04-25T00:56:20.236465+0000","last_fullsized":"2024-04-25T00:56:20.236465+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:30.756244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272390+0000","last_change":"2024-04-25T00:56:19.260505+0000","last_active":"2024-04-25T00:56:20.272390+0000","last_peered":"2024-04-25T00:56:20.272390+0000","last_clean":"2024-04-25T00:56:20.272390+0000","last_became_active":"2024-04-25T00:56:19.257992+0000","last_became_peered":"2024-04-25T00:56:19.257992+0000","last_unstale":"2024-04-25T00:56:20.272390+0000","last_undegraded":"2024-04-25T00:56:20.272390+0000","last_fullsized":"2024-04-25T00:56:20.272390+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:53.956326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.234461+0000","last_change":"2024-04-25T00:56:19.259498+0000","last_active":"2024-04-25T00:56:20.234461+0000","last_peered":"2024-04-25T00:56:20.234461+0000","last_clean":"2024-04-25T00:56:20.234461+0000","last_became_active":"2024-04-25T00:56:19.252811+0000","last_became_peered":"2024-04-25T00:56:19.252811+0000","last_unstale":"2024-04-25T00:56:20.234461+0000","last_undegraded":"2024-04-25T00:56:20.234461+0000","last_fullsized":"2024-04-25T00:56:20.234461+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:09.582451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275768+0000","last_change":"2024-04-25T00:56:19.268774+0000","last_active":"2024-04-25T00:56:20.275768+0000","last_peered":"2024-04-25T00:56:20.275768+0000","last_clean":"2024-04-25T00:56:20.275768+0000","last_became_active":"2024-04-25T00:56:19.267867+0000","last_became_peered":"2024-04-25T00:56:19.267867+0000","last_unstale":"2024-04-25T00:56:20.275768+0000","last_undegraded":"2024-04-25T00:56:20.275768+0000","last_fullsized":"2024-04-25T00:56:20.275768+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:12.328808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.027035+0000","last_change":"2024-04-25T00:56:19.272184+0000","last_active":"2024-04-25T00:56:21.027035+0000","last_peered":"2024-04-25T00:56:21.027035+0000","last_clean":"2024-04-25T00:56:21.027035+0000","last_became_active":"2024-04-25T00:56:19.271713+0000","last_became_peered":"2024-04-25T00:56:19.271713+0000","last_unstale":"2024-04-25T00:56:21.027035+0000","last_undegraded":"2024-04-25T00:56:21.027035+0000","last_fullsized":"2024-04-25T00:56:21.027035+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:06.049756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275823+0000","last_change":"2024-04-25T00:56:19.274973+0000","last_active":"2024-04-25T00:56:20.275823+0000","last_peered":"2024-04-25T00:56:20.275823+0000","last_clean":"2024-04-25T00:56:20.275823+0000","last_became_active":"2024-04-25T00:56:19.274629+0000","last_became_peered":"2024-04-25T00:56:19.274629+0000","last_unstale":"2024-04-25T00:56:20.275823+0000","last_undegraded":"2024-04-25T00:56:20.275823+0000","last_fullsized":"2024-04-25T00:56:20.275823+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:09.582451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236293+0000","last_change":"2024-04-25T00:56:19.259559+0000","last_active":"2024-04-25T00:56:20.236293+0000","last_peered":"2024-04-25T00:56:20.236293+0000","last_clean":"2024-04-25T00:56:20.236293+0000","last_became_active":"2024-04-25T00:56:19.247149+0000","last_became_peered":"2024-04-25T00:56:19.247149+0000","last_unstale":"2024-04-25T00:56:20.236293+0000","last_undegraded":"2024-04-25T00:56:20.236293+0000","last_fullsized":"2024-04-25T00:56:20.236293+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:31.737001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.235396+0000","last_change":"2024-04-25T00:56:19.270582+0000","last_active":"2024-04-25T00:56:20.235396+0000","last_peered":"2024-04-25T00:56:20.235396+0000","last_clean":"2024-04-25T00:56:20.235396+0000","last_became_active":"2024-04-25T00:56:19.269694+0000","last_became_peered":"2024-04-25T00:56:19.269694+0000","last_unstale":"2024-04-25T00:56:20.235396+0000","last_undegraded":"2024-04-25T00:56:20.235396+0000","last_fullsized":"2024-04-25T00:56:20.235396+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:45.897182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025042+0000","last_change":"2024-04-25T00:56:19.282960+0000","last_active":"2024-04-25T00:56:21.025042+0000","last_peered":"2024-04-25T00:56:21.025042+0000","last_clean":"2024-04-25T00:56:21.025042+0000","last_became_active":"2024-04-25T00:56:19.282196+0000","last_became_peered":"2024-04-25T00:56:19.282196+0000","last_unstale":"2024-04-25T00:56:21.025042+0000","last_undegraded":"2024-04-25T00:56:21.025042+0000","last_fullsized":"2024-04-25T00:56:21.025042+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:13:09.582451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026130+0000","last_change":"2024-04-25T00:56:19.280622+0000","last_active":"2024-04-25T00:56:21.026130+0000","last_peered":"2024-04-25T00:56:21.026130+0000","last_clean":"2024-04-25T00:56:21.026130+0000","last_became_active":"2024-04-25T00:56:19.279471+0000","last_became_peered":"2024-04-25T00:56:19.279471+0000","last_unstale":"2024-04-25T00:56:21.026130+0000","last_undegraded":"2024-04-25T00:56:21.026130+0000","last_fullsized":"2024-04-25T00:56:21.026130+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:53.956326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.024764+0000","last_change":"2024-04-25T00:56:19.266298+0000","last_active":"2024-04-25T00:56:21.024764+0000","last_peered":"2024-04-25T00:56:21.024764+0000","last_clean":"2024-04-25T00:56:21.024764+0000","last_became_active":"2024-04-25T00:56:19.265531+0000","last_became_peered":"2024-04-25T00:56:19.265531+0000","last_unstale":"2024-04-25T00:56:21.024764+0000","last_undegraded":"2024-04-25T00:56:21.024764+0000","last_fullsized":"2024-04-25T00:56:21.024764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:00.230340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272664+0000","last_change":"2024-04-25T00:56:19.258825+0000","last_active":"2024-04-25T00:56:20.272664+0000","last_peered":"2024-04-25T00:56:20.272664+0000","last_clean":"2024-04-25T00:56:20.272664+0000","last_became_active":"2024-04-25T00:56:19.258103+0000","last_became_peered":"2024-04-25T00:56:19.258103+0000","last_unstale":"2024-04-25T00:56:20.272664+0000","last_undegraded":"2024-04-25T00:56:20.272664+0000","last_fullsized":"2024-04-25T00:56:20.272664+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:45.897182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"28'4","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.027288+0000","last_change":"2024-04-25T00:56:18.270923+0000","last_active":"2024-04-25T00:56:21.027288+0000","last_peered":"2024-04-25T00:56:21.027288+0000","last_clean":"2024-04-25T00:56:21.027288+0000","last_became_active":"2024-04-25T00:56:18.270126+0000","last_became_peered":"2024-04-25T00:56:18.270126+0000","last_unstale":"2024-04-25T00:56:21.027288+0000","last_undegraded":"2024-04-25T00:56:21.027288+0000","last_fullsized":"2024-04-25T00:56:21.027288+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:09.793618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.027224+0000","last_change":"2024-04-25T00:56:19.280473+0000","last_active":"2024-04-25T00:56:21.027224+0000","last_peered":"2024-04-25T00:56:21.027224+0000","last_clean":"2024-04-25T00:56:21.027224+0000","last_became_active":"2024-04-25T00:56:19.279873+0000","last_became_peered":"2024-04-25T00:56:19.279873+0000","last_unstale":"2024-04-25T00:56:21.027224+0000","last_undegraded":"2024-04-25T00:56:21.027224+0000","last_fullsized":"2024-04-25T00:56:21.027224+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:57:30.756244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272638+0000","last_change":"2024-04-25T00:56:18.282797+0000","last_active":"2024-04-25T00:56:20.272638+0000","last_peered":"2024-04-25T00:56:20.272638+0000","last_clean":"2024-04-25T00:56:20.272638+0000","last_became_active":"2024-04-25T00:56:18.282436+0000","last_became_peered":"2024-04-25T00:56:18.282436+0000","last_unstale":"2024-04-25T00:56:20.272638+0000","last_undegraded":"2024-04-25T00:56:20.272638+0000","last_fullsized":"2024-04-25T00:56:20.272638+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:21.336081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.238269+0000","last_change":"2024-04-25T00:56:19.272783+0000","last_active":"2024-04-25T00:56:20.238269+0000","last_peered":"2024-04-25T00:56:20.238269+0000","last_clean":"2024-04-25T00:56:20.238269+0000","last_became_active":"2024-04-25T00:56:19.272054+0000","last_became_peered":"2024-04-25T00:56:19.272054+0000","last_unstale":"2024-04-25T00:56:20.238269+0000","last_undegraded":"2024-04-25T00:56:20.238269+0000","last_fullsized":"2024-04-25T00:56:20.238269+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:47.799770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026055+0000","last_change":"2024-04-25T00:56:18.263463+0000","last_active":"2024-04-25T00:56:21.026055+0000","last_peered":"2024-04-25T00:56:21.026055+0000","last_clean":"2024-04-25T00:56:21.026055+0000","last_became_active":"2024-04-25T00:56:18.263018+0000","last_became_peered":"2024-04-25T00:56:18.263018+0000","last_unstale":"2024-04-25T00:56:21.026055+0000","last_undegraded":"2024-04-25T00:56:21.026055+0000","last_fullsized":"2024-04-25T00:56:21.026055+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:02.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275903+0000","last_change":"2024-04-25T00:56:19.269144+0000","last_active":"2024-04-25T00:56:20.275903+0000","last_peered":"2024-04-25T00:56:20.275903+0000","last_clean":"2024-04-25T00:56:20.275903+0000","last_became_active":"2024-04-25T00:56:19.268770+0000","last_became_peered":"2024-04-25T00:56:19.268770+0000","last_unstale":"2024-04-25T00:56:20.275903+0000","last_undegraded":"2024-04-25T00:56:20.275903+0000","last_fullsized":"2024-04-25T00:56:20.275903+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:08.938239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236531+0000","last_change":"2024-04-25T00:56:18.259673+0000","last_active":"2024-04-25T00:56:20.236531+0000","last_peered":"2024-04-25T00:56:20.236531+0000","last_clean":"2024-04-25T00:56:20.236531+0000","last_became_active":"2024-04-25T00:56:18.259261+0000","last_became_peered":"2024-04-25T00:56:18.259261+0000","last_unstale":"2024-04-25T00:56:20.236531+0000","last_undegraded":"2024-04-25T00:56:20.236531+0000","last_fullsized":"2024-04-25T00:56:20.236531+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:37.028135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025460+0000","last_change":"2024-04-25T00:56:19.265956+0000","last_active":"2024-04-25T00:56:21.025460+0000","last_peered":"2024-04-25T00:56:21.025460+0000","last_clean":"2024-04-25T00:56:21.025460+0000","last_became_active":"2024-04-25T00:56:19.265069+0000","last_became_peered":"2024-04-25T00:56:19.265069+0000","last_unstale":"2024-04-25T00:56:21.025460+0000","last_undegraded":"2024-04-25T00:56:21.025460+0000","last_fullsized":"2024-04-25T00:56:21.025460+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:53.822998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.237348+0000","last_change":"2024-04-25T00:56:18.268494+0000","last_active":"2024-04-25T00:56:20.237348+0000","last_peered":"2024-04-25T00:56:20.237348+0000","last_clean":"2024-04-25T00:56:20.237348+0000","last_became_active":"2024-04-25T00:56:18.266110+0000","last_became_peered":"2024-04-25T00:56:18.266110+0000","last_unstale":"2024-04-25T00:56:20.237348+0000","last_undegraded":"2024-04-25T00:56:20.237348+0000","last_fullsized":"2024-04-25T00:56:20.237348+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:21.296462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.235239+0000","last_change":"2024-04-25T00:56:19.259585+0000","last_active":"2024-04-25T00:56:20.235239+0000","last_peered":"2024-04-25T00:56:20.235239+0000","last_clean":"2024-04-25T00:56:20.235239+0000","last_became_active":"2024-04-25T00:56:19.247114+0000","last_became_peered":"2024-04-25T00:56:19.247114+0000","last_unstale":"2024-04-25T00:56:20.235239+0000","last_undegraded":"2024-04-25T00:56:20.235239+0000","last_fullsized":"2024-04-25T00:56:20.235239+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:08.938239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236624+0000","last_change":"2024-04-25T00:56:18.267152+0000","last_active":"2024-04-25T00:56:20.236624+0000","last_peered":"2024-04-25T00:56:20.236624+0000","last_clean":"2024-04-25T00:56:20.236624+0000","last_became_active":"2024-04-25T00:56:18.266214+0000","last_became_peered":"2024-04-25T00:56:18.266214+0000","last_unstale":"2024-04-25T00:56:20.236624+0000","last_undegraded":"2024-04-25T00:56:20.236624+0000","last_fullsized":"2024-04-25T00:56:20.236624+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:58:39.371183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.234759+0000","last_change":"2024-04-25T00:56:19.282025+0000","last_active":"2024-04-25T00:56:20.234759+0000","last_peered":"2024-04-25T00:56:20.234759+0000","last_clean":"2024-04-25T00:56:20.234759+0000","last_became_active":"2024-04-25T00:56:19.281302+0000","last_became_peered":"2024-04-25T00:56:19.281302+0000","last_unstale":"2024-04-25T00:56:20.234759+0000","last_undegraded":"2024-04-25T00:56:20.234759+0000","last_fullsized":"2024-04-25T00:56:20.234759+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:53.956326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025305+0000","last_change":"2024-04-25T00:56:18.252064+0000","last_active":"2024-04-25T00:56:21.025305+0000","last_peered":"2024-04-25T00:56:21.025305+0000","last_clean":"2024-04-25T00:56:21.025305+0000","last_became_active":"2024-04-25T00:56:18.251551+0000","last_became_peered":"2024-04-25T00:56:18.251551+0000","last_unstale":"2024-04-25T00:56:21.025305+0000","last_undegraded":"2024-04-25T00:56:21.025305+0000","last_fullsized":"2024-04-25T00:56:21.025305+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:26.983336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275451+0000","last_change":"2024-04-25T00:56:19.271072+0000","last_active":"2024-04-25T00:56:20.275451+0000","last_peered":"2024-04-25T00:56:20.275451+0000","last_clean":"2024-04-25T00:56:20.275451+0000","last_became_active":"2024-04-25T00:56:19.270686+0000","last_became_peered":"2024-04-25T00:56:19.270686+0000","last_unstale":"2024-04-25T00:56:20.275451+0000","last_undegraded":"2024-04-25T00:56:20.275451+0000","last_fullsized":"2024-04-25T00:56:20.275451+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:49:47.799770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.234890+0000","last_change":"2024-04-25T00:56:18.278855+0000","last_active":"2024-04-25T00:56:20.234890+0000","last_peered":"2024-04-25T00:56:20.234890+0000","last_clean":"2024-04-25T00:56:20.234890+0000","last_became_active":"2024-04-25T00:56:18.278043+0000","last_became_peered":"2024-04-25T00:56:18.278043+0000","last_unstale":"2024-04-25T00:56:20.234890+0000","last_undegraded":"2024-04-25T00:56:20.234890+0000","last_fullsized":"2024-04-25T00:56:20.234890+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:32.786597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236913+0000","last_change":"2024-04-25T00:56:19.274855+0000","last_active":"2024-04-25T00:56:20.236913+0000","last_peered":"2024-04-25T00:56:20.236913+0000","last_clean":"2024-04-25T00:56:20.236913+0000","last_became_active":"2024-04-25T00:56:19.274079+0000","last_became_peered":"2024-04-25T00:56:19.274079+0000","last_unstale":"2024-04-25T00:56:20.236913+0000","last_undegraded":"2024-04-25T00:56:20.236913+0000","last_fullsized":"2024-04-25T00:56:20.236913+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:53.822998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026622+0000","last_change":"2024-04-25T00:56:18.282682+0000","last_active":"2024-04-25T00:56:21.026622+0000","last_peered":"2024-04-25T00:56:21.026622+0000","last_clean":"2024-04-25T00:56:21.026622+0000","last_became_active":"2024-04-25T00:56:18.282301+0000","last_became_peered":"2024-04-25T00:56:18.282301+0000","last_unstale":"2024-04-25T00:56:21.026622+0000","last_undegraded":"2024-04-25T00:56:21.026622+0000","last_fullsized":"2024-04-25T00:56:21.026622+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:52.951374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026658+0000","last_change":"2024-04-25T00:56:19.269699+0000","last_active":"2024-04-25T00:56:21.026658+0000","last_peered":"2024-04-25T00:56:21.026658+0000","last_clean":"2024-04-25T00:56:21.026658+0000","last_became_active":"2024-04-25T00:56:19.269338+0000","last_became_peered":"2024-04-25T00:56:19.269338+0000","last_unstale":"2024-04-25T00:56:21.026658+0000","last_undegraded":"2024-04-25T00:56:21.026658+0000","last_fullsized":"2024-04-25T00:56:21.026658+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:13.472791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272704+0000","last_change":"2024-04-25T00:56:18.254931+0000","last_active":"2024-04-25T00:56:20.272704+0000","last_peered":"2024-04-25T00:56:20.272704+0000","last_clean":"2024-04-25T00:56:20.272704+0000","last_became_active":"2024-04-25T00:56:18.254550+0000","last_became_peered":"2024-04-25T00:56:18.254550+0000","last_unstale":"2024-04-25T00:56:20.272704+0000","last_undegraded":"2024-04-25T00:56:20.272704+0000","last_fullsized":"2024-04-25T00:56:20.272704+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:09.793618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026605+0000","last_change":"2024-04-25T00:56:19.271075+0000","last_active":"2024-04-25T00:56:21.026605+0000","last_peered":"2024-04-25T00:56:21.026605+0000","last_clean":"2024-04-25T00:56:21.026605+0000","last_became_active":"2024-04-25T00:56:19.270780+0000","last_became_peered":"2024-04-25T00:56:19.270780+0000","last_unstale":"2024-04-25T00:56:21.026605+0000","last_undegraded":"2024-04-25T00:56:21.026605+0000","last_fullsized":"2024-04-25T00:56:21.026605+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:49.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"28'3","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.237734+0000","last_change":"2024-04-25T00:56:18.273909+0000","last_active":"2024-04-25T00:56:20.237734+0000","last_peered":"2024-04-25T00:56:20.237734+0000","last_clean":"2024-04-25T00:56:20.237734+0000","last_became_active":"2024-04-25T00:56:18.273187+0000","last_became_peered":"2024-04-25T00:56:18.273187+0000","last_unstale":"2024-04-25T00:56:20.237734+0000","last_undegraded":"2024-04-25T00:56:20.237734+0000","last_fullsized":"2024-04-25T00:56:20.237734+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:52.951374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.237673+0000","last_change":"2024-04-25T00:56:19.268628+0000","last_active":"2024-04-25T00:56:20.237673+0000","last_peered":"2024-04-25T00:56:20.237673+0000","last_clean":"2024-04-25T00:56:20.237673+0000","last_became_active":"2024-04-25T00:56:19.267785+0000","last_became_peered":"2024-04-25T00:56:19.267785+0000","last_unstale":"2024-04-25T00:56:20.237673+0000","last_undegraded":"2024-04-25T00:56:20.237673+0000","last_fullsized":"2024-04-25T00:56:20.237673+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:16:09.392942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"28'6","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026059+0000","last_change":"2024-04-25T00:56:18.265249+0000","last_active":"2024-04-25T00:56:21.026059+0000","last_peered":"2024-04-25T00:56:21.026059+0000","last_clean":"2024-04-25T00:56:21.026059+0000","last_became_active":"2024-04-25T00:56:18.265006+0000","last_became_peered":"2024-04-25T00:56:18.265006+0000","last_unstale":"2024-04-25T00:56:21.026059+0000","last_undegraded":"2024-04-25T00:56:21.026059+0000","last_fullsized":"2024-04-25T00:56:21.026059+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:40.440523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275467+0000","last_change":"2024-04-25T00:56:19.271012+0000","last_active":"2024-04-25T00:56:20.275467+0000","last_peered":"2024-04-25T00:56:20.275467+0000","last_clean":"2024-04-25T00:56:20.275467+0000","last_became_active":"2024-04-25T00:56:19.270673+0000","last_became_peered":"2024-04-25T00:56:19.270673+0000","last_unstale":"2024-04-25T00:56:20.275467+0000","last_undegraded":"2024-04-25T00:56:20.275467+0000","last_fullsized":"2024-04-25T00:56:20.275467+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:39.872224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.237384+0000","last_change":"2024-04-25T00:56:18.275099+0000","last_active":"2024-04-25T00:56:20.237384+0000","last_peered":"2024-04-25T00:56:20.237384+0000","last_clean":"2024-04-25T00:56:20.237384+0000","last_became_active":"2024-04-25T00:56:18.274111+0000","last_became_peered":"2024-04-25T00:56:18.274111+0000","last_unstale":"2024-04-25T00:56:20.237384+0000","last_undegraded":"2024-04-25T00:56:20.237384+0000","last_fullsized":"2024-04-25T00:56:20.237384+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:21.336081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026380+0000","last_change":"2024-04-25T00:56:19.269301+0000","last_active":"2024-04-25T00:56:21.026380+0000","last_peered":"2024-04-25T00:56:21.026380+0000","last_clean":"2024-04-25T00:56:21.026380+0000","last_became_active":"2024-04-25T00:56:19.268834+0000","last_became_peered":"2024-04-25T00:56:19.268834+0000","last_unstale":"2024-04-25T00:56:21.026380+0000","last_undegraded":"2024-04-25T00:56:21.026380+0000","last_fullsized":"2024-04-25T00:56:21.026380+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:07.700486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275225+0000","last_change":"2024-04-25T00:56:18.272779+0000","last_active":"2024-04-25T00:56:20.275225+0000","last_peered":"2024-04-25T00:56:20.275225+0000","last_clean":"2024-04-25T00:56:20.275225+0000","last_became_active":"2024-04-25T00:56:18.272371+0000","last_became_peered":"2024-04-25T00:56:18.272371+0000","last_unstale":"2024-04-25T00:56:20.275225+0000","last_undegraded":"2024-04-25T00:56:20.275225+0000","last_fullsized":"2024-04-25T00:56:20.275225+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:41.664660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025032+0000","last_change":"2024-04-25T00:56:19.264646+0000","last_active":"2024-04-25T00:56:21.025032+0000","last_peered":"2024-04-25T00:56:21.025032+0000","last_clean":"2024-04-25T00:56:21.025032+0000","last_became_active":"2024-04-25T00:56:19.263910+0000","last_became_peered":"2024-04-25T00:56:19.263910+0000","last_unstale":"2024-04-25T00:56:21.025032+0000","last_undegraded":"2024-04-25T00:56:21.025032+0000","last_fullsized":"2024-04-25T00:56:21.025032+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:52.367492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025473+0000","last_change":"2024-04-25T00:56:18.276094+0000","last_active":"2024-04-25T00:56:21.025473+0000","last_peered":"2024-04-25T00:56:21.025473+0000","last_clean":"2024-04-25T00:56:21.025473+0000","last_became_active":"2024-04-25T00:56:18.275798+0000","last_became_peered":"2024-04-25T00:56:18.275798+0000","last_unstale":"2024-04-25T00:56:21.025473+0000","last_undegraded":"2024-04-25T00:56:21.025473+0000","last_fullsized":"2024-04-25T00:56:21.025473+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:22:44.892229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026444+0000","last_change":"2024-04-25T00:56:19.283904+0000","last_active":"2024-04-25T00:56:21.026444+0000","last_peered":"2024-04-25T00:56:21.026444+0000","last_clean":"2024-04-25T00:56:21.026444+0000","last_became_active":"2024-04-25T00:56:19.279909+0000","last_became_peered":"2024-04-25T00:56:19.279909+0000","last_unstale":"2024-04-25T00:56:21.026444+0000","last_undegraded":"2024-04-25T00:56:21.026444+0000","last_fullsized":"2024-04-25T00:56:21.026444+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:41.445476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"28'8","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025380+0000","last_change":"2024-04-25T00:56:18.259582+0000","last_active":"2024-04-25T00:56:21.025380+0000","last_peered":"2024-04-25T00:56:21.025380+0000","last_clean":"2024-04-25T00:56:21.025380+0000","last_became_active":"2024-04-25T00:56:18.259229+0000","last_became_peered":"2024-04-25T00:56:18.259229+0000","last_unstale":"2024-04-25T00:56:21.025380+0000","last_undegraded":"2024-04-25T00:56:21.025380+0000","last_fullsized":"2024-04-25T00:56:21.025380+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:41.606166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.238491+0000","last_change":"2024-04-25T00:56:19.243670+0000","last_active":"2024-04-25T00:56:20.238491+0000","last_peered":"2024-04-25T00:56:20.238491+0000","last_clean":"2024-04-25T00:56:20.238491+0000","last_became_active":"2024-04-25T00:56:19.242904+0000","last_became_peered":"2024-04-25T00:56:19.242904+0000","last_unstale":"2024-04-25T00:56:20.238491+0000","last_undegraded":"2024-04-25T00:56:20.238491+0000","last_fullsized":"2024-04-25T00:56:20.238491+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:35:39.872224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275734+0000","last_change":"2024-04-25T00:56:18.261994+0000","last_active":"2024-04-25T00:56:20.275734+0000","last_peered":"2024-04-25T00:56:20.275734+0000","last_clean":"2024-04-25T00:56:20.275734+0000","last_became_active":"2024-04-25T00:56:18.261587+0000","last_became_peered":"2024-04-25T00:56:18.261587+0000","last_unstale":"2024-04-25T00:56:20.275734+0000","last_undegraded":"2024-04-25T00:56:20.275734+0000","last_fullsized":"2024-04-25T00:56:20.275734+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:21.336081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.238864+0000","last_change":"2024-04-25T00:56:19.245060+0000","last_active":"2024-04-25T00:56:20.238864+0000","last_peered":"2024-04-25T00:56:20.238864+0000","last_clean":"2024-04-25T00:56:20.238864+0000","last_became_active":"2024-04-25T00:56:19.244316+0000","last_became_peered":"2024-04-25T00:56:19.244316+0000","last_unstale":"2024-04-25T00:56:20.238864+0000","last_undegraded":"2024-04-25T00:56:20.238864+0000","last_fullsized":"2024-04-25T00:56:20.238864+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:07.700486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.238493+0000","last_change":"2024-04-25T00:56:18.274873+0000","last_active":"2024-04-25T00:56:20.238493+0000","last_peered":"2024-04-25T00:56:20.238493+0000","last_clean":"2024-04-25T00:56:20.238493+0000","last_became_active":"2024-04-25T00:56:18.274061+0000","last_became_peered":"2024-04-25T00:56:18.274061+0000","last_unstale":"2024-04-25T00:56:20.238493+0000","last_undegraded":"2024-04-25T00:56:20.238493+0000","last_fullsized":"2024-04-25T00:56:20.238493+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:47.637824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275827+0000","last_change":"2024-04-25T00:56:19.265498+0000","last_active":"2024-04-25T00:56:20.275827+0000","last_peered":"2024-04-25T00:56:20.275827+0000","last_clean":"2024-04-25T00:56:20.275827+0000","last_became_active":"2024-04-25T00:56:19.264742+0000","last_became_peered":"2024-04-25T00:56:19.264742+0000","last_unstale":"2024-04-25T00:56:20.275827+0000","last_undegraded":"2024-04-25T00:56:20.275827+0000","last_fullsized":"2024-04-25T00:56:20.275827+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:29:52.367492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"28'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.238901+0000","last_change":"2024-04-25T00:56:18.267235+0000","last_active":"2024-04-25T00:56:20.238901+0000","last_peered":"2024-04-25T00:56:20.238901+0000","last_clean":"2024-04-25T00:56:20.238901+0000","last_became_active":"2024-04-25T00:56:18.266347+0000","last_became_peered":"2024-04-25T00:56:18.266347+0000","last_unstale":"2024-04-25T00:56:20.238901+0000","last_undegraded":"2024-04-25T00:56:20.238901+0000","last_fullsized":"2024-04-25T00:56:20.238901+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:04.689772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236952+0000","last_change":"2024-04-25T00:56:19.258374+0000","last_active":"2024-04-25T00:56:20.236952+0000","last_peered":"2024-04-25T00:56:20.236952+0000","last_clean":"2024-04-25T00:56:20.236952+0000","last_became_active":"2024-04-25T00:56:19.256570+0000","last_became_peered":"2024-04-25T00:56:19.256570+0000","last_unstale":"2024-04-25T00:56:20.236952+0000","last_undegraded":"2024-04-25T00:56:20.236952+0000","last_fullsized":"2024-04-25T00:56:20.236952+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:14:22.341033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.238048+0000","last_change":"2024-04-25T00:56:18.268874+0000","last_active":"2024-04-25T00:56:20.238048+0000","last_peered":"2024-04-25T00:56:20.238048+0000","last_clean":"2024-04-25T00:56:20.238048+0000","last_became_active":"2024-04-25T00:56:18.267243+0000","last_became_peered":"2024-04-25T00:56:18.267243+0000","last_unstale":"2024-04-25T00:56:20.238048+0000","last_undegraded":"2024-04-25T00:56:20.238048+0000","last_fullsized":"2024-04-25T00:56:20.238048+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:37.998221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236279+0000","last_change":"2024-04-25T00:56:19.248435+0000","last_active":"2024-04-25T00:56:20.236279+0000","last_peered":"2024-04-25T00:56:20.236279+0000","last_clean":"2024-04-25T00:56:20.236279+0000","last_became_active":"2024-04-25T00:56:19.247680+0000","last_became_peered":"2024-04-25T00:56:19.247680+0000","last_unstale":"2024-04-25T00:56:20.236279+0000","last_undegraded":"2024-04-25T00:56:20.236279+0000","last_fullsized":"2024-04-25T00:56:20.236279+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:05.694725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272480+0000","last_change":"2024-04-25T00:56:18.263622+0000","last_active":"2024-04-25T00:56:20.272480+0000","last_peered":"2024-04-25T00:56:20.272480+0000","last_clean":"2024-04-25T00:56:20.272480+0000","last_became_active":"2024-04-25T00:56:18.263263+0000","last_became_peered":"2024-04-25T00:56:18.263263+0000","last_unstale":"2024-04-25T00:56:20.272480+0000","last_undegraded":"2024-04-25T00:56:20.272480+0000","last_fullsized":"2024-04-25T00:56:20.272480+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:21.296462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236732+0000","last_change":"2024-04-25T00:56:19.254521+0000","last_active":"2024-04-25T00:56:20.236732+0000","last_peered":"2024-04-25T00:56:20.236732+0000","last_clean":"2024-04-25T00:56:20.236732+0000","last_became_active":"2024-04-25T00:56:19.253759+0000","last_became_peered":"2024-04-25T00:56:19.253759+0000","last_unstale":"2024-04-25T00:56:20.236732+0000","last_undegraded":"2024-04-25T00:56:20.236732+0000","last_fullsized":"2024-04-25T00:56:20.236732+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:38:48.642777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026675+0000","last_change":"2024-04-25T00:56:18.271035+0000","last_active":"2024-04-25T00:56:21.026675+0000","last_peered":"2024-04-25T00:56:21.026675+0000","last_clean":"2024-04-25T00:56:21.026675+0000","last_became_active":"2024-04-25T00:56:18.270284+0000","last_became_peered":"2024-04-25T00:56:18.270284+0000","last_unstale":"2024-04-25T00:56:21.026675+0000","last_undegraded":"2024-04-25T00:56:21.026675+0000","last_fullsized":"2024-04-25T00:56:21.026675+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:06.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.235884+0000","last_change":"2024-04-25T00:56:19.253821+0000","last_active":"2024-04-25T00:56:20.235884+0000","last_peered":"2024-04-25T00:56:20.235884+0000","last_clean":"2024-04-25T00:56:20.235884+0000","last_became_active":"2024-04-25T00:56:19.253085+0000","last_became_peered":"2024-04-25T00:56:19.253085+0000","last_unstale":"2024-04-25T00:56:20.235884+0000","last_undegraded":"2024-04-25T00:56:20.235884+0000","last_fullsized":"2024-04-25T00:56:20.235884+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:03.106366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.235289+0000","last_change":"2024-04-25T00:56:18.268167+0000","last_active":"2024-04-25T00:56:20.235289+0000","last_peered":"2024-04-25T00:56:20.235289+0000","last_clean":"2024-04-25T00:56:20.235289+0000","last_became_active":"2024-04-25T00:56:18.266648+0000","last_became_peered":"2024-04-25T00:56:18.266648+0000","last_unstale":"2024-04-25T00:56:20.235289+0000","last_undegraded":"2024-04-25T00:56:20.235289+0000","last_fullsized":"2024-04-25T00:56:20.235289+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:39:02.101413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.024822+0000","last_change":"2024-04-25T00:56:19.282386+0000","last_active":"2024-04-25T00:56:21.024822+0000","last_peered":"2024-04-25T00:56:21.024822+0000","last_clean":"2024-04-25T00:56:21.024822+0000","last_became_active":"2024-04-25T00:56:19.281834+0000","last_became_peered":"2024-04-25T00:56:19.281834+0000","last_unstale":"2024-04-25T00:56:21.024822+0000","last_undegraded":"2024-04-25T00:56:21.024822+0000","last_fullsized":"2024-04-25T00:56:21.024822+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:12.328808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.234138+0000","last_change":"2024-04-25T00:56:18.282738+0000","last_active":"2024-04-25T00:56:20.234138+0000","last_peered":"2024-04-25T00:56:20.234138+0000","last_clean":"2024-04-25T00:56:20.234138+0000","last_became_active":"2024-04-25T00:56:18.281996+0000","last_became_peered":"2024-04-25T00:56:18.281996+0000","last_unstale":"2024-04-25T00:56:20.234138+0000","last_undegraded":"2024-04-25T00:56:20.234138+0000","last_fullsized":"2024-04-25T00:56:20.234138+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:52.042475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.236845+0000","last_change":"2024-04-25T00:56:19.248306+0000","last_active":"2024-04-25T00:56:20.236845+0000","last_peered":"2024-04-25T00:56:20.236845+0000","last_clean":"2024-04-25T00:56:20.236845+0000","last_became_active":"2024-04-25T00:56:19.247504+0000","last_became_peered":"2024-04-25T00:56:19.247504+0000","last_unstale":"2024-04-25T00:56:20.236845+0000","last_undegraded":"2024-04-25T00:56:20.236845+0000","last_fullsized":"2024-04-25T00:56:20.236845+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:17:22.301415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"28'2","reported_seq":16,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275839+0000","last_change":"2024-04-25T00:56:18.264899+0000","last_active":"2024-04-25T00:56:20.275839+0000","last_peered":"2024-04-25T00:56:20.275839+0000","last_clean":"2024-04-25T00:56:20.275839+0000","last_became_active":"2024-04-25T00:56:18.264520+0000","last_became_peered":"2024-04-25T00:56:18.264520+0000","last_unstale":"2024-04-25T00:56:20.275839+0000","last_undegraded":"2024-04-25T00:56:20.275839+0000","last_fullsized":"2024-04-25T00:56:20.275839+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:09.793618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.239251+0000","last_change":"2024-04-25T00:56:19.268525+0000","last_active":"2024-04-25T00:56:20.239251+0000","last_peered":"2024-04-25T00:56:20.239251+0000","last_clean":"2024-04-25T00:56:20.239251+0000","last_became_active":"2024-04-25T00:56:19.267827+0000","last_became_peered":"2024-04-25T00:56:19.267827+0000","last_unstale":"2024-04-25T00:56:20.239251+0000","last_undegraded":"2024-04-25T00:56:20.239251+0000","last_fullsized":"2024-04-25T00:56:20.239251+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:49.631330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"29'1","reported_seq":15,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272814+0000","last_change":"2024-04-25T00:56:18.263505+0000","last_active":"2024-04-25T00:56:20.272814+0000","last_peered":"2024-04-25T00:56:20.272814+0000","last_clean":"2024-04-25T00:56:20.272814+0000","last_became_active":"2024-04-25T00:56:18.263059+0000","last_became_peered":"2024-04-25T00:56:18.263059+0000","last_unstale":"2024-04-25T00:56:20.272814+0000","last_undegraded":"2024-04-25T00:56:20.272814+0000","last_fullsized":"2024-04-25T00:56:20.272814+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:06.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272547+0000","last_change":"2024-04-25T00:56:19.267311+0000","last_active":"2024-04-25T00:56:20.272547+0000","last_peered":"2024-04-25T00:56:20.272547+0000","last_clean":"2024-04-25T00:56:20.272547+0000","last_became_active":"2024-04-25T00:56:19.267000+0000","last_became_peered":"2024-04-25T00:56:19.267000+0000","last_unstale":"2024-04-25T00:56:20.272547+0000","last_undegraded":"2024-04-25T00:56:20.272547+0000","last_fullsized":"2024-04-25T00:56:20.272547+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:27.988288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275832+0000","last_change":"2024-04-25T00:56:18.259289+0000","last_active":"2024-04-25T00:56:20.275832+0000","last_peered":"2024-04-25T00:56:20.275832+0000","last_clean":"2024-04-25T00:56:20.275832+0000","last_became_active":"2024-04-25T00:56:18.258925+0000","last_became_peered":"2024-04-25T00:56:18.258925+0000","last_unstale":"2024-04-25T00:56:20.275832+0000","last_undegraded":"2024-04-25T00:56:20.275832+0000","last_fullsized":"2024-04-25T00:56:20.275832+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:06.697389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026878+0000","last_change":"2024-04-25T00:56:19.279011+0000","last_active":"2024-04-25T00:56:21.026878+0000","last_peered":"2024-04-25T00:56:21.026878+0000","last_clean":"2024-04-25T00:56:21.026878+0000","last_became_active":"2024-04-25T00:56:19.278259+0000","last_became_peered":"2024-04-25T00:56:19.278259+0000","last_unstale":"2024-04-25T00:56:21.026878+0000","last_undegraded":"2024-04-25T00:56:21.026878+0000","last_fullsized":"2024-04-25T00:56:21.026878+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:27.988288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272635+0000","last_change":"2024-04-25T00:56:18.262868+0000","last_active":"2024-04-25T00:56:20.272635+0000","last_peered":"2024-04-25T00:56:20.272635+0000","last_clean":"2024-04-25T00:56:20.272635+0000","last_became_active":"2024-04-25T00:56:18.262492+0000","last_became_peered":"2024-04-25T00:56:18.262492+0000","last_unstale":"2024-04-25T00:56:20.272635+0000","last_undegraded":"2024-04-25T00:56:20.272635+0000","last_fullsized":"2024-04-25T00:56:20.272635+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:41.664660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.272546+0000","last_change":"2024-04-25T00:56:19.281325+0000","last_active":"2024-04-25T00:56:20.272546+0000","last_peered":"2024-04-25T00:56:20.272546+0000","last_clean":"2024-04-25T00:56:20.272546+0000","last_became_active":"2024-04-25T00:56:19.281033+0000","last_became_peered":"2024-04-25T00:56:19.281033+0000","last_unstale":"2024-04-25T00:56:20.272546+0000","last_undegraded":"2024-04-25T00:56:20.272546+0000","last_fullsized":"2024-04-25T00:56:20.272546+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:31:41.445476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.275365+0000","last_change":"2024-04-25T00:56:18.262139+0000","last_active":"2024-04-25T00:56:20.275365+0000","last_peered":"2024-04-25T00:56:20.275365+0000","last_clean":"2024-04-25T00:56:20.275365+0000","last_became_active":"2024-04-25T00:56:18.261800+0000","last_became_peered":"2024-04-25T00:56:18.261800+0000","last_unstale":"2024-04-25T00:56:20.275365+0000","last_undegraded":"2024-04-25T00:56:20.275365+0000","last_fullsized":"2024-04-25T00:56:20.275365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:26:56.849533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025380+0000","last_change":"2024-04-25T00:56:19.271153+0000","last_active":"2024-04-25T00:56:21.025380+0000","last_peered":"2024-04-25T00:56:21.025380+0000","last_clean":"2024-04-25T00:56:21.025380+0000","last_became_active":"2024-04-25T00:56:19.270760+0000","last_became_peered":"2024-04-25T00:56:19.270760+0000","last_unstale":"2024-04-25T00:56:21.025380+0000","last_undegraded":"2024-04-25T00:56:21.025380+0000","last_fullsized":"2024-04-25T00:56:21.025380+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:25:53.822998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.235534+0000","last_change":"2024-04-25T00:56:18.268057+0000","last_active":"2024-04-25T00:56:20.235534+0000","last_peered":"2024-04-25T00:56:20.235534+0000","last_clean":"2024-04-25T00:56:20.235534+0000","last_became_active":"2024-04-25T00:56:18.266732+0000","last_became_peered":"2024-04-25T00:56:18.266732+0000","last_unstale":"2024-04-25T00:56:20.235534+0000","last_undegraded":"2024-04-25T00:56:20.235534+0000","last_fullsized":"2024-04-25T00:56:20.235534+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:19.789402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:20.235145+0000","last_change":"2024-04-25T00:56:19.272432+0000","last_active":"2024-04-25T00:56:20.235145+0000","last_peered":"2024-04-25T00:56:20.235145+0000","last_clean":"2024-04-25T00:56:20.235145+0000","last_became_active":"2024-04-25T00:56:19.271763+0000","last_became_peered":"2024-04-25T00:56:19.271763+0000","last_unstale":"2024-04-25T00:56:20.235145+0000","last_undegraded":"2024-04-25T00:56:20.235145+0000","last_fullsized":"2024-04-25T00:56:20.235145+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:27.988288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025527+0000","last_change":"2024-04-25T00:56:18.282683+0000","last_active":"2024-04-25T00:56:21.025527+0000","last_peered":"2024-04-25T00:56:21.025527+0000","last_clean":"2024-04-25T00:56:21.025527+0000","last_became_active":"2024-04-25T00:56:18.282229+0000","last_became_peered":"2024-04-25T00:56:18.282229+0000","last_unstale":"2024-04-25T00:56:21.025527+0000","last_undegraded":"2024-04-25T00:56:21.025527+0000","last_fullsized":"2024-04-25T00:56:21.025527+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:52.042475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025398+0000","last_change":"2024-04-25T00:56:19.272446+0000","last_active":"2024-04-25T00:56:21.025398+0000","last_peered":"2024-04-25T00:56:21.025398+0000","last_clean":"2024-04-25T00:56:21.025398+0000","last_became_active":"2024-04-25T00:56:19.271450+0000","last_became_peered":"2024-04-25T00:56:19.271450+0000","last_unstale":"2024-04-25T00:56:21.025398+0000","last_undegraded":"2024-04-25T00:56:21.025398+0000","last_fullsized":"2024-04-25T00:56:21.025398+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:18.221586+0000","last_clean_scrub_stamp":"2024-04-25T00:56:18.221586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:31.737001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.026648+0000","last_change":"2024-04-25T00:56:18.276834+0000","last_active":"2024-04-25T00:56:21.026648+0000","last_peered":"2024-04-25T00:56:21.026648+0000","last_clean":"2024-04-25T00:56:21.026648+0000","last_became_active":"2024-04-25T00:56:18.276075+0000","last_became_peered":"2024-04-25T00:56:18.276075+0000","last_unstale":"2024-04-25T00:56:21.026648+0000","last_undegraded":"2024-04-25T00:56:21.026648+0000","last_fullsized":"2024-04-25T00:56:21.026648+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:56:17.216633+0000","last_clean_scrub_stamp":"2024-04-25T00:56:17.216633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:51:41.664660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025163+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:56:21.025163+0000","last_peered":"2024-04-25T00:56:21.025163+0000","last_clean":"2024-04-25T00:56:21.025163+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:56:21.025163+0000","last_undegraded":"2024-04-25T00:56:21.025163+0000","last_fullsized":"2024-04-25T00:56:21.025163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":125646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":124082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":125646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":714380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:56:23.602 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:56:23.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T00:56:23.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf2484d640 1 Processor -- start 2024-04-25T00:56:23.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf2484d640 1 -- start start 2024-04-25T00:56:23.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf2484d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdf2006d1b0 0x7fdf2013e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf2484d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 0x7fdf2013e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf2484d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf200af950 0x7fdf20144460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf2484d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fdf200b5230 con 0x7fdf200af950 2024-04-25T00:56:23.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf2484d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdf200b4c20 con 0x7fdf2006d1b0 2024-04-25T00:56:23.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf2484d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2006d590 con 0x7fdf200b2370 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf200af950 0x7fdf20144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdf200af950 0x7fdf20144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57724/0 (socket says 172.21.15.112:57724) 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1ed9d640 1 -- 172.21.15.112:0/3689994591 learned_addr learned my addr 172.21.15.112:0/3689994591 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1e59c640 1 --2- 172.21.15.112:0/3689994591 >> v2:172.21.15.187:3300/0 conn(0x7fdf2006d1b0 0x7fdf2013e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1dd9b640 1 --2- 172.21.15.112:0/3689994591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 0x7fdf2013e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1ed9d640 1 -- 172.21.15.112:0/3689994591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 msgr2=0x7fdf2013e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1ed9d640 1 --2- 172.21.15.112:0/3689994591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 0x7fdf2013e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1ed9d640 1 -- 172.21.15.112:0/3689994591 >> v2:172.21.15.187:3300/0 conn(0x7fdf2006d1b0 msgr2=0x7fdf2013e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1ed9d640 1 --2- 172.21.15.112:0/3689994591 >> v2:172.21.15.187:3300/0 conn(0x7fdf2006d1b0 0x7fdf2013e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.763+0000 7fdf1ed9d640 1 -- 172.21.15.112:0/3689994591 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2006f360 con 0x7fdf200af950 2024-04-25T00:56:23.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1ed9d640 1 --2- 172.21.15.112:0/3689994591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf20144460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdf2006d870 tx=0x7fdf1801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a5061311cfd56c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1d59a640 1 -- 172.21.15.112:0/3689994591 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180a5740 con 0x7fdf200af950 2024-04-25T00:56:23.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1d59a640 1 -- 172.21.15.112:0/3689994591 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf180a58e0 con 0x7fdf200af950 2024-04-25T00:56:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1d59a640 1 -- 172.21.15.112:0/3689994591 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180a5cb0 con 0x7fdf200af950 2024-04-25T00:56:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- 172.21.15.112:0/3689994591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 msgr2=0x7fdf20144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.767 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3689994591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf20144460 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdf2006d870 tx=0x7fdf1801c9f0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- 172.21.15.112:0/3689994591 shutdown_connections 2024-04-25T00:56:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3689994591 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf20144460 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3689994591 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 0x7fdf2013e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3689994591 >> v2:172.21.15.187:3300/0 conn(0x7fdf2006d1b0 0x7fdf2013e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- 172.21.15.112:0/3689994591 >> 172.21.15.112:0/3689994591 conn(0x7fdf200b5540 msgr2=0x7fdf200b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- 172.21.15.112:0/3689994591 shutdown_connections 2024-04-25T00:56:23.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- 172.21.15.112:0/3689994591 wait complete. 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 Processor -- start 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- start start 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf2006d1b0 0x7fdf20140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf200b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 0x7fdf2013ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20148220 con 0x7fdf200af950 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201480a0 con 0x7fdf2006d1b0 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdf201483a0 con 0x7fdf200b2370 2024-04-25T00:56:23.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf200b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf200b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57726/0 (socket says 172.21.15.112:57726) 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 -- 172.21.15.112:0/3261654676 learned_addr learned my addr 172.21.15.112:0/3261654676 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1ed9d640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 0x7fdf2013ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1e59c640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf2006d1b0 0x7fdf20140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 -- 172.21.15.112:0/3261654676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 msgr2=0x7fdf2013ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 0x7fdf2013ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 -- 172.21.15.112:0/3261654676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf2006d1b0 msgr2=0x7fdf20140200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf2006d1b0 0x7fdf20140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 -- 172.21.15.112:0/3261654676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2014aa90 con 0x7fdf200af950 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf1dd9b640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf200b3ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d800 tx=0x7fdf1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf0effd640 1 -- 172.21.15.112:0/3261654676 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400b6c0 con 0x7fdf200af950 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf0effd640 1 -- 172.21.15.112:0/3261654676 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf14019070 con 0x7fdf200af950 2024-04-25T00:56:23.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf0effd640 1 -- 172.21.15.112:0/3261654676 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1400baa0 con 0x7fdf200af950 2024-04-25T00:56:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf20072b50 con 0x7fdf200af950 2024-04-25T00:56:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.767+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf20148aa0 con 0x7fdf200af950 2024-04-25T00:56:23.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.771+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf20072d50 con 0x7fdf200af950 2024-04-25T00:56:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.771+0000 7fdf0effd640 1 -- 172.21.15.112:0/3261654676 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf1400bc40 con 0x7fdf200af950 2024-04-25T00:56:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.771+0000 7fdf0effd640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdf000678c0 0x7fdf00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.771+0000 7fdf0effd640 1 -- 172.21.15.112:0/3261654676 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fdf14086030 con 0x7fdf200af950 2024-04-25T00:56:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.771+0000 7fdf1e59c640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdf000678c0 0x7fdf00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:23.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.771+0000 7fdf1e59c640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdf000678c0 0x7fdf00069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf100096f0 tx=0x7fdf10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:23.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.775+0000 7fdf0effd640 1 -- 172.21.15.112:0/3261654676 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdf2014aa90 con 0x7fdf200af950 2024-04-25T00:56:23.996 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.995+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdf20068260 con 0x7fdf200af950 2024-04-25T00:56:24.000 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf0effd640 1 -- 172.21.15.112:0/3261654676 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7fdf14056c00 con 0x7fdf200af950 2024-04-25T00:56:24.000 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2024-04-25T00:56:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDRqSlmLHGXDBAAcY5ixa+8Gv2MeY7OW7Jffw== 2024-04-25T00:56:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: caps mds = "allow" 2024-04-25T00:56:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: caps mon = "allow r" 2024-04-25T00:56:24.001 INFO:teuthology.orchestra.run.smithi112.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T00:56:24.003 INFO:teuthology.orchestra.run.smithi112.stderr:updated caps for client.0 2024-04-25T00:56:24.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdf000678c0 msgr2=0x7fdf00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdf000678c0 0x7fdf00069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdf100096f0 tx=0x7fdf10008040 comp rx=0 tx=0).stop 2024-04-25T00:56:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 msgr2=0x7fdf200b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf200b3ae0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdf1400d800 tx=0x7fdf1400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 shutdown_connections 2024-04-25T00:56:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdf200b2370 0x7fdf2013ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdf200af950 0x7fdf200b3ae0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdf000678c0 0x7fdf00069d80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 --2- 172.21.15.112:0/3261654676 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdf2006d1b0 0x7fdf20140200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 >> 172.21.15.112:0/3261654676 conn(0x7fdf200b5540 msgr2=0x7fdf200afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 shutdown_connections 2024-04-25T00:56:24.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:23.999+0000 7fdf2484d640 1 -- 172.21.15.112:0/3261654676 wait complete. 2024-04-25T00:56:24.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:24.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d462b2640 1 Processor -- start 2024-04-25T00:56:24.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d462b2640 1 -- start start 2024-04-25T00:56:24.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d462b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400745b0 0x7f5d401487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d462b2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5d40142b80 0x7f5d400b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d462b2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d40074990 con 0x7f5d400b3b40 2024-04-25T00:56:24.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d462b2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f5d4009ca30 con 0x7f5d40142b80 2024-04-25T00:56:24.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d462b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d400b9cc0 con 0x7f5d400745b0 2024-04-25T00:56:24.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d4484f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5d40142b80 0x7f5d400b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.125 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d4484f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5d40142b80 0x7f5d400b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57730/0 (socket says 172.21.15.112:57730) 2024-04-25T00:56:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d4484f640 1 -- 172.21.15.112:0/1276350079 learned_addr learned my addr 172.21.15.112:0/1276350079 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d3f7fe640 1 --1- 172.21.15.112:0/1276350079 >> v1:172.21.15.112:6789/0 conn(0x7f5d400b3b40 0x7f5d40142470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58928/0 (socket says 172.21.15.112:58928) 2024-04-25T00:56:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d3ffff640 1 --2- 172.21.15.112:0/1276350079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400745b0 0x7f5d401487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d4484f640 1 -- 172.21.15.112:0/1276350079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400745b0 msgr2=0x7f5d401487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d4484f640 1 --2- 172.21.15.112:0/1276350079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400745b0 0x7f5d401487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d4484f640 1 -- 172.21.15.112:0/1276350079 >> v1:172.21.15.112:6789/0 conn(0x7f5d400b3b40 legacy=0x7f5d40142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d4484f640 1 -- 172.21.15.112:0/1276350079 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d400b48c0 con 0x7f5d40142b80 2024-04-25T00:56:24.126 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.122+0000 7f5d3ffff640 1 --2- 172.21.15.112:0/1276350079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400745b0 0x7f5d401487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:24.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d4484f640 1 --2- 172.21.15.112:0/1276350079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d400b4de0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009e50 tx=0x7f5d3001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76edeb87204dec42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.127 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d3effd640 1 -- 172.21.15.112:0/1276350079 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d300a5880 con 0x7f5d40142b80 2024-04-25T00:56:24.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d3effd640 1 -- 172.21.15.112:0/1276350079 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d300a5a20 con 0x7f5d40142b80 2024-04-25T00:56:24.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d3effd640 1 -- 172.21.15.112:0/1276350079 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d300a5df0 con 0x7f5d40142b80 2024-04-25T00:56:24.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 -- 172.21.15.112:0/1276350079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 msgr2=0x7f5d400b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.128 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 --2- 172.21.15.112:0/1276350079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d400b4de0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5d30009e50 tx=0x7f5d3001c9a0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 -- 172.21.15.112:0/1276350079 shutdown_connections 2024-04-25T00:56:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 --2- 172.21.15.112:0/1276350079 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d400b4de0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 --2- 172.21.15.112:0/1276350079 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400745b0 0x7f5d401487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 -- 172.21.15.112:0/1276350079 >> 172.21.15.112:0/1276350079 conn(0x7f5d400b6240 msgr2=0x7f5d400b9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 -- 172.21.15.112:0/1276350079 shutdown_connections 2024-04-25T00:56:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 -- 172.21.15.112:0/1276350079 wait complete. 2024-04-25T00:56:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 Processor -- start 2024-04-25T00:56:24.129 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.126+0000 7f5d462b2640 1 -- start start 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d400745b0 0x7f5d40140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400b3b40 0x7f5d4006fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d4009e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40152480 con 0x7f5d40142b80 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5d40152300 con 0x7f5d400745b0 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5d40152600 con 0x7f5d400b3b40 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d4009e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d4009e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57742/0 (socket says 172.21.15.112:57742) 2024-04-25T00:56:24.132 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 -- 172.21.15.112:0/426237938 learned_addr learned my addr 172.21.15.112:0/426237938 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3f7fe640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400b3b40 0x7f5d4006fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3ffff640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d400745b0 0x7f5d40140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 -- 172.21.15.112:0/426237938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400b3b40 msgr2=0x7f5d4006fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400b3b40 0x7f5d4006fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 -- 172.21.15.112:0/426237938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d400745b0 msgr2=0x7f5d40140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d400745b0 0x7f5d40140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 -- 172.21.15.112:0/426237938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d40072320 con 0x7f5d40142b80 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d4484f640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d4009e200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5d300af040 tx=0x7f5d30019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3cff9640 1 -- 172.21.15.112:0/426237938 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30009250 con 0x7f5d40142b80 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3cff9640 1 -- 172.21.15.112:0/426237938 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d300b8650 con 0x7f5d40142b80 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d40071660 con 0x7f5d40142b80 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3cff9640 1 -- 172.21.15.112:0/426237938 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d300b8a40 con 0x7f5d40142b80 2024-04-25T00:56:24.133 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d400725c0 con 0x7f5d40142b80 2024-04-25T00:56:24.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d4004f580 con 0x7f5d40142b80 2024-04-25T00:56:24.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3cff9640 1 -- 172.21.15.112:0/426237938 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5d300c1020 con 0x7f5d40142b80 2024-04-25T00:56:24.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3cff9640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5d2c0678f0 0x7f5d2c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3cff9640 1 -- 172.21.15.112:0/426237938 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f5d30120340 con 0x7f5d40142b80 2024-04-25T00:56:24.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3ffff640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5d2c0678f0 0x7f5d2c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.135 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.130+0000 7f5d3ffff640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5d2c0678f0 0x7f5d2c069db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.137 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.134+0000 7f5d3cff9640 1 -- 172.21.15.112:0/426237938 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5d300c8020 con 0x7f5d40142b80 2024-04-25T00:56:24.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d40067da0 con 0x7f5d40142b80 2024-04-25T00:56:24.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d3cff9640 1 -- 172.21.15.112:0/426237938 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3537 (secure 0 0 0) 0x7f5d30016620 con 0x7f5d40142b80 2024-04-25T00:56:24.320 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:24.320 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3898281312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3898281312},{"type":"v1","addr":"172.21.15.187:6833","nonce":3898281312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:56:18.588797+0000","modified":"2024-04-25T00:56:21.588160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3222796217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3222796217},{"type":"v1","addr":"172.21.15.112:6835","nonce":3222796217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-04-25T00:56:24.320 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-04-25T00:56:24.322 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5d2c0678f0 msgr2=0x7f5d2c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5d2c0678f0 0x7f5d2c069db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5d38006fd0 tx=0x7f5d38008040 comp rx=0 tx=0).stop 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 msgr2=0x7f5d4009e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d4009e200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5d300af040 tx=0x7f5d30019740 comp rx=0 tx=0).stop 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 shutdown_connections 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5d40142b80 0x7f5d4009e200 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5d400b3b40 0x7f5d4006fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f5d2c0678f0 0x7f5d2c069db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 --2- 172.21.15.112:0/426237938 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5d400745b0 0x7f5d40140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 >> 172.21.15.112:0/426237938 conn(0x7f5d400b6240 msgr2=0x7f5d40140f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 shutdown_connections 2024-04-25T00:56:24.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.318+0000 7f5d462b2640 1 -- 172.21.15.112:0/426237938 wait complete. 2024-04-25T00:56:24.539 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:56:18.588797+0000', 'modified': '2024-04-25T00:56:21.588160+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4926}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4926': {'gid': 4926, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6835/3222796217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 3222796217}, {'type': 'v1', 'addr': '172.21.15.112:6835', 'nonce': 3222796217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4926, 'qdb_cluster': [4926]} 2024-04-25T00:56:24.539 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:56:24.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:56:24.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7976e36640 1 Processor -- start 2024-04-25T00:56:24.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7976e36640 1 -- start start 2024-04-25T00:56:24.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7976e36640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f797006c630 0x7f79700aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.666 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7976e36640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f79700add70 0x7f79700b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7976e36640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79700af760 con 0x7f797006ca10 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7976e36640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7970148860 con 0x7f797006c630 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7976e36640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f79700b52f0 con 0x7f79700add70 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7974bd2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f797006c630 0x7f79700aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7974bd2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f797006c630 0x7f79700aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57754/0 (socket says 172.21.15.112:57754) 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f7974bd2640 1 -- 172.21.15.112:0/1854726122 learned_addr learned my addr 172.21.15.112:0/1854726122 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.662+0000 7f79753d3640 1 --2- 172.21.15.112:0/1854726122 >> v2:172.21.15.187:3300/0 conn(0x7f79700add70 0x7f79700b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7974bd2640 1 -- 172.21.15.112:0/1854726122 >> v2:172.21.15.187:3300/0 conn(0x7f79700add70 msgr2=0x7f79700b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7974bd2640 1 --2- 172.21.15.112:0/1854726122 >> v2:172.21.15.187:3300/0 conn(0x7f79700add70 0x7f79700b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7974bd2640 1 -- 172.21.15.112:0/1854726122 >> v1:172.21.15.187:6789/0 conn(0x7f797006ca10 legacy=0x7f79701477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7974bd2640 1 -- 172.21.15.112:0/1854726122 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79700afa00 con 0x7f797006c630 2024-04-25T00:56:24.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7974bd2640 1 --2- 172.21.15.112:0/1854726122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006c630 0x7f79700aff80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f79700b9bc0 tx=0x7f796001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b95d9cf67c6e4e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f796f7fe640 1 -- 172.21.15.112:0/1854726122 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79600b0070 con 0x7f797006c630 2024-04-25T00:56:24.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f796f7fe640 1 -- 172.21.15.112:0/1854726122 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79600ab600 con 0x7f797006c630 2024-04-25T00:56:24.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f796f7fe640 1 -- 172.21.15.112:0/1854726122 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79600ab9d0 con 0x7f797006c630 2024-04-25T00:56:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7976e36640 1 -- 172.21.15.112:0/1854726122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006c630 msgr2=0x7f79700aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7976e36640 1 --2- 172.21.15.112:0/1854726122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006c630 0x7f79700aff80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f79700b9bc0 tx=0x7f796001c3c0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7976e36640 1 -- 172.21.15.112:0/1854726122 shutdown_connections 2024-04-25T00:56:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7976e36640 1 --2- 172.21.15.112:0/1854726122 >> v2:172.21.15.187:3300/0 conn(0x7f79700add70 0x7f79700b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7976e36640 1 --2- 172.21.15.112:0/1854726122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006c630 0x7f79700aff80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7976e36640 1 -- 172.21.15.112:0/1854726122 >> 172.21.15.112:0/1854726122 conn(0x7f79700b5980 msgr2=0x7f79700b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7976e36640 1 -- 172.21.15.112:0/1854726122 shutdown_connections 2024-04-25T00:56:24.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.666+0000 7f7976e36640 1 -- 172.21.15.112:0/1854726122 wait complete. 2024-04-25T00:56:24.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 Processor -- start 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 -- start start 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f797006c630 0x7f7970146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006ca10 0x7f7970142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79700add70 0x7f79701e3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79701519f0 con 0x7f797006ca10 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7970151870 con 0x7f797006c630 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7970151b70 con 0x7f79700add70 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006ca10 0x7f7970142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006ca10 0x7f7970142820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57770/0 (socket says 172.21.15.112:57770) 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 -- 172.21.15.112:0/947687298 learned_addr learned my addr 172.21.15.112:0/947687298 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f79753d3640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79700add70 0x7f79701e3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7974bd2640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f797006c630 0x7f7970146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 -- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79700add70 msgr2=0x7f79701e3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79700add70 0x7f79701e3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 -- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f797006c630 msgr2=0x7f7970146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f797006c630 0x7f7970146230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 -- 172.21.15.112:0/947687298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7970143ef0 con 0x7f797006ca10 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7974bd2640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f797006c630 0x7f7970146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796ffff640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006ca10 0x7f7970142820 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f796400c410 tx=0x7f796400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796d7fa640 1 -- 172.21.15.112:0/947687298 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964011790 con 0x7f797006ca10 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796d7fa640 1 -- 172.21.15.112:0/947687298 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7964017070 con 0x7f797006ca10 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796d7fa640 1 -- 172.21.15.112:0/947687298 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964011b70 con 0x7f797006ca10 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 -- 172.21.15.112:0/947687298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7970070cd0 con 0x7f797006ca10 2024-04-25T00:56:24.674 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7976e36640 1 -- 172.21.15.112:0/947687298 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7970070fb0 con 0x7f797006ca10 2024-04-25T00:56:24.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796d7fa640 1 -- 172.21.15.112:0/947687298 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7964011d10 con 0x7f797006ca10 2024-04-25T00:56:24.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796d7fa640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f795c0678c0 0x7f795c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f796d7fa640 1 -- 172.21.15.112:0/947687298 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f79640858e0 con 0x7f797006ca10 2024-04-25T00:56:24.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.670+0000 7f7974bd2640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f795c0678c0 0x7f795c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f795c0678c0 0x7f795c069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79600ad000 tx=0x7f7960008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 Processor -- start 2024-04-25T00:56:24.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- start start 2024-04-25T00:56:24.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197dd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 0x55e4197ddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e4195fb5e0 con 0x55e4197ded90 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e419582c50 con 0x55e4197dd850 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e4195f75a0 con 0x55e4197b3140 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57778/0 (socket says 172.21.15.112:57778) 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 -- 172.21.15.112:0/3463292274 learned_addr learned my addr 172.21.15.112:0/3463292274 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f79753d3640 1 --2- 172.21.15.112:0/3463292274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 0x55e4197ddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 -- 172.21.15.112:0/3463292274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 msgr2=0x55e4197dd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 --2- 172.21.15.112:0/3463292274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197dd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 -- 172.21.15.112:0/3463292274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 msgr2=0x55e4197ddc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.682 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 --2- 172.21.15.112:0/3463292274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 0x55e4197ddc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 -- 172.21.15.112:0/3463292274 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e419580d10 con 0x55e4197ded90 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7974bd2640 1 --2- 172.21.15.112:0/3463292274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197b2c00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f79600ae040 tx=0x7f7960004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7cb0651e50555c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f79567fc640 1 -- 172.21.15.112:0/3463292274 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79600c6070 con 0x55e4197ded90 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f79567fc640 1 -- 172.21.15.112:0/3463292274 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79600abde0 con 0x55e4197ded90 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f79567fc640 1 -- 172.21.15.112:0/3463292274 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79600b0040 con 0x55e4197ded90 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- 172.21.15.112:0/3463292274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 msgr2=0x55e4197b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- 172.21.15.112:0/3463292274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197b2c00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f79600ae040 tx=0x7f7960004740 comp rx=0 tx=0).stop 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- 172.21.15.112:0/3463292274 shutdown_connections 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- 172.21.15.112:0/3463292274 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 0x55e4197ddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- 172.21.15.112:0/3463292274 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197dd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- 172.21.15.112:0/3463292274 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197b2c00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- 172.21.15.112:0/3463292274 >> 172.21.15.112:0/3463292274 conn(0x55e4197d90c0 msgr2=0x55e4197db4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- 172.21.15.112:0/3463292274 shutdown_connections 2024-04-25T00:56:24.683 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- 172.21.15.112:0/3463292274 wait complete. 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- start start 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 Processor -- start 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 0x55e4197af230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e4195ffc10 con 0x55e4197dd850 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e4195b5520 con 0x55e4197ded90 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e4195c8340 con 0x55e4197b3140 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f7979faf000 10 client.0 _ll_register_callbacks cb 0x55e4197985c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f796ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f796ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57782/0 (socket says 172.21.15.112:57782) 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.674+0000 7f796ffff640 1 -- 172.21.15.112:0/1135842784 learned_addr learned my addr 172.21.15.112:0/1135842784 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7974bd2640 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.684 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f79753d3640 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 0x55e4197af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f796ffff640 1 -- 172.21.15.112:0/1135842784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 msgr2=0x55e4197b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f796ffff640 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f796ffff640 1 -- 172.21.15.112:0/1135842784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 msgr2=0x55e4197af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f796ffff640 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 0x55e4197af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f796ffff640 1 -- 172.21.15.112:0/1135842784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4195937b0 con 0x55e4197dd850 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f796ffff640 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197aecf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f796405e3d0 tx=0x7f7964055e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b146446533523739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 1 -- 172.21.15.112:0/1135842784 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964055170 con 0x55e4197dd850 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 -- 172.21.15.112:0/1135842784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79600abde0 con 0x55e4197dd850 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 1 -- 172.21.15.112:0/1135842784 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f7964055310 con 0x55e4197dd850 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 1 -- 172.21.15.112:0/1135842784 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f79640740d0 con 0x55e4197dd850 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 10 client.5094 fetch_fsmap learned FSMap version 16 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 -- 172.21.15.112:0/1135842784 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55e419582ea0 con 0x55e4197dd850 2024-04-25T00:56:24.685 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 1 -- 172.21.15.112:0/1135842784 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79600abde0 con 0x55e4197dd850 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 1 -- 172.21.15.112:0/1135842784 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 16) ==== 2284+0+0 (secure 0 0 0) 0x7f796406d020 con 0x55e4197dd850 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 10 client.5094 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 10 client.5094 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 20 client.5094 populate_metadata read hostname 'smithi112' 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] conn(0x55e4197af770 0x55e4197f4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 4 client.5094 mds_command: new command op to 4926 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 -- 172.21.15.112:0/1135842784 --> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e419596ab0 con 0x55e4197af770 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7974bd2640 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] conn(0x55e4197af770 0x55e4197f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7974bd2640 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] conn(0x55e4197af770 0x55e4197f4560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf98abdf221830e4 server_cookie=849b9e8b771f8a64 in_seq=0 out_seq=0 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 10 client.5094 ms_handle_connect on v2:172.21.15.112:6834/3222796217 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 1 -- 172.21.15.112:0/1135842784 <== mds.0 v2:172.21.15.112:6834/3222796217 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e419596ab0 con 0x55e4197af770 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 10 client.5094 handle_command_reply: tid=0 2024-04-25T00:56:24.686 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 client.5094 shutdown 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 -- 172.21.15.112:0/1135842784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 msgr2=0x55e4197aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197aecf0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f796405e3d0 tx=0x7f7964055e60 comp rx=0 tx=0).stop 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 -- 172.21.15.112:0/1135842784 shutdown_connections 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] conn(0x55e4197af770 0x55e4197f4560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 0x55e4197af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f794f7fe640 0 client.5094 ms_handle_reset on v2:172.21.15.112:6834/3222796217 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197aecf0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 --2- 172.21.15.112:0/1135842784 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 -- 172.21.15.112:0/1135842784 >> 172.21.15.112:0/1135842784 conn(0x55e4197d90c0 msgr2=0x55e4197db6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 -- 172.21.15.112:0/1135842784 shutdown_connections 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 1 -- 172.21.15.112:0/1135842784 wait complete. 2024-04-25T00:56:24.687 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.678+0000 7f7979faf000 20 client.5094 trim_cache size 0 max 16384 2024-04-25T00:56:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.694+0000 7f7979faf000 1 Processor -- start 2024-04-25T00:56:24.698 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.694+0000 7f7979faf000 1 -- start start 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197dd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 0x55e4197f7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f79701ecc60 con 0x55e4197dd850 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f79701ebdf0 con 0x55e4197ded90 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e4195fa8f0 con 0x55e4197b3140 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197d82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57798/0 (socket says 172.21.15.112:57798) 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 -- 172.21.15.112:0/978318372 learned_addr learned my addr 172.21.15.112:0/978318372 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 -- 172.21.15.112:0/978318372 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 msgr2=0x55e4197dd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 --2- 172.21.15.112:0/978318372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 0x55e4197f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 --2- 172.21.15.112:0/978318372 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197dd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.701 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 -- 172.21.15.112:0/978318372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 msgr2=0x55e4197f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 --2- 172.21.15.112:0/978318372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 0x55e4197f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 -- 172.21.15.112:0/978318372 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e419583e80 con 0x55e4197dd850 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 --2- 172.21.15.112:0/978318372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197d82a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7964000c10 tx=0x7f7964013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926db9467989039f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7955ffb640 1 -- 172.21.15.112:0/978318372 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964052390 con 0x55e4197dd850 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7955ffb640 1 -- 172.21.15.112:0/978318372 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7964079070 con 0x55e4197dd850 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7955ffb640 1 -- 172.21.15.112:0/978318372 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7964052530 con 0x55e4197dd850 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- 172.21.15.112:0/978318372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 msgr2=0x55e4197d82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- 172.21.15.112:0/978318372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197d82a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7964000c10 tx=0x7f7964013000 comp rx=0 tx=0).stop 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- 172.21.15.112:0/978318372 shutdown_connections 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- 172.21.15.112:0/978318372 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197ded90 0x55e4197f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- 172.21.15.112:0/978318372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197dd850 0x55e4197d82a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- 172.21.15.112:0/978318372 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197dd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.702 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- 172.21.15.112:0/978318372 >> 172.21.15.112:0/978318372 conn(0x55e4197d90c0 msgr2=0x55e4197d7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- 172.21.15.112:0/978318372 shutdown_connections 2024-04-25T00:56:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- 172.21.15.112:0/978318372 wait complete. 2024-04-25T00:56:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- start start 2024-04-25T00:56:24.703 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 Processor -- start 2024-04-25T00:56:24.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.706 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 0x55e4197f7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x55e4197efc90 con 0x55e4197ded90 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x55e4195ffc10 con 0x55e4197dd850 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x55e419654850 con 0x55e4197b3140 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 10 client.0 _ll_register_callbacks cb 0x7f7964051fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197f7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57814/0 (socket says 172.21.15.112:57814) 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 -- 172.21.15.112:0/800414035 learned_addr learned my addr 172.21.15.112:0/800414035 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f796ffff640 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 0x55e4197f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 -- 172.21.15.112:0/800414035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 msgr2=0x55e4197aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7974bd2640 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 -- 172.21.15.112:0/800414035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 msgr2=0x55e4197f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.707 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 0x55e4197f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 -- 172.21.15.112:0/800414035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4195c8760 con 0x55e4197ded90 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f79753d3640 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197f7ad0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7968007980 tx=0x7f7968007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe341a8224cb42b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f794f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.112:3300/0 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f794f7fe640 1 -- 172.21.15.112:0/800414035 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796801a070 con 0x55e4197ded90 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f794f7fe640 1 -- 172.21.15.112:0/800414035 <== mon.0 v2:172.21.15.112:3300/0 2 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f7968012c30 con 0x55e4197ded90 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- 172.21.15.112:0/800414035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7964079070 con 0x55e4197ded90 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f794f7fe640 1 -- 172.21.15.112:0/800414035 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f7968025050 con 0x55e4197ded90 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 10 client.5100 fetch_fsmap learned FSMap version 16 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.698+0000 7f7979faf000 1 -- 172.21.15.112:0/800414035 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55e4196549e0 con 0x55e4197ded90 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f794f7fe640 1 -- 172.21.15.112:0/800414035 <== mon.0 v2:172.21.15.112:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7964079070 con 0x55e4197ded90 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f794f7fe640 1 -- 172.21.15.112:0/800414035 <== mon.0 v2:172.21.15.112:3300/0 5 ==== fsmap(e 16) ==== 2284+0+0 (secure 0 0 0) 0x7f7968028020 con 0x55e4197ded90 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 10 client.5100 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 10 client.5100 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:56:24.708 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 20 client.5100 populate_metadata read hostname 'smithi112' 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] conn(0x55e4197f4560 0x55e4197f4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 4 client.5100 mds_command: new command op to 4926 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 -- 172.21.15.112:0/800414035 --> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] -- command(tid 0: {"prefix": "status"}) -- 0x55e419608b20 con 0x55e4197f4560 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7974bd2640 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] conn(0x55e4197f4560 0x55e4197f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f794f7fe640 10 client.5100 ms_handle_connect on v2:172.21.15.112:6834/3222796217 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7974bd2640 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] conn(0x55e4197f4560 0x55e4197f4940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48bf49c86a0e3ead server_cookie=92dd7bacca51a933 in_seq=0 out_seq=0 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f794f7fe640 1 -- 172.21.15.112:0/800414035 <== mds.0 v2:172.21.15.112:6834/3222796217 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e419608b20 con 0x55e4197f4560 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f794f7fe640 10 client.5100 handle_command_reply: tid=0 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 client.5100 shutdown 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 -- 172.21.15.112:0/800414035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 msgr2=0x55e4197f7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197f7ad0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7968007980 tx=0x7f7968007e40 comp rx=0 tx=0).stop 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 -- 172.21.15.112:0/800414035 shutdown_connections 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x55e4197ded90 0x55e4197f7ad0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.709 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.112:6834/3222796217,v1:172.21.15.112:6835/3222796217] conn(0x55e4197f4560 0x55e4197f4940 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x55e4197dd850 0x55e4197f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 --2- 172.21.15.112:0/800414035 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x55e4197b3140 0x55e4197aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f794f7fe640 0 client.5100 ms_handle_reset on v2:172.21.15.112:6834/3222796217 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 -- 172.21.15.112:0/800414035 >> 172.21.15.112:0/800414035 conn(0x55e4197d90c0 msgr2=0x55e4197d8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 -- 172.21.15.112:0/800414035 shutdown_connections 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 1 -- 172.21.15.112:0/800414035 wait complete. 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7979faf000 20 client.5100 trim_cache size 0 max 16384 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "cluster_fsid": "86f864a3-be50-4513-98ca-86783428a0f8", 2024-04-25T00:56:24.710 INFO:teuthology.orchestra.run.smithi112.stdout: "whoami": 0, 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4926, 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "want_state": "up:active", 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "state": "up:active", 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs", 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "rank_uptime": 5.4405957840000001, 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "mdsmap_epoch": 16, 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch": 29, 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "osdmap_epoch_barrier": 29, 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout: "uptime": 12.541883156999999 2024-04-25T00:56:24.711 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 -- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f795c0678c0 msgr2=0x7f795c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.712 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f795c0678c0 0x7f795c069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79600ad000 tx=0x7f7960008040 comp rx=0 tx=0).stop 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 -- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006ca10 msgr2=0x7f7970142820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006ca10 0x7f7970142820 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f796400c410 tx=0x7f796400c8d0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 -- 172.21.15.112:0/947687298 shutdown_connections 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f79700add70 0x7f79701e3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f797006ca10 0x7f7970142820 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f795c0678c0 0x7f795c069d80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 --2- 172.21.15.112:0/947687298 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f797006c630 0x7f7970146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 -- 172.21.15.112:0/947687298 >> 172.21.15.112:0/947687298 conn(0x7f79700b5980 msgr2=0x7f797013f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 -- 172.21.15.112:0/947687298 shutdown_connections 2024-04-25T00:56:24.713 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.702+0000 7f7976e36640 1 -- 172.21.15.112:0/947687298 wait complete. 2024-04-25T00:56:24.720 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T00:56:24.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.826+0000 7f216fe3c640 1 Processor -- start 2024-04-25T00:56:24.830 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.826+0000 7f216fe3c640 1 -- start start 2024-04-25T00:56:24.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216fe3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f216806c630 0x7f21680aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216fe3c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f21680add70 0x7f21680b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216fe3c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21680af760 con 0x7f216806ca10 2024-04-25T00:56:24.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216fe3c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f2168148860 con 0x7f21680add70 2024-04-25T00:56:24.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216fe3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f21680b52f0 con 0x7f216806c630 2024-04-25T00:56:24.831 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216d3d7640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f216806ca10 0x7f21681477c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58942/0 (socket says 172.21.15.112:58942) 2024-04-25T00:56:24.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216d3d7640 1 -- 172.21.15.112:0/4045026016 learned_addr learned my addr 172.21.15.112:0/4045026016 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:24.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216dbd8640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f216806c630 0x7f21680aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216e3d9640 1 --2- 172.21.15.112:0/4045026016 >> v2:172.21.15.187:3300/0 conn(0x7f21680add70 0x7f21680b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1156164774 0 0) 0x7f21680af760 con 0x7f216806ca10 2024-04-25T00:56:24.832 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2150005740 con 0x7f216806ca10 2024-04-25T00:56:24.833 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 429716227 0 0) 0x7f2150005740 con 0x7f216806ca10 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f216806c630 msgr2=0x7f21680aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f216806c630 0x7f21680aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 >> v2:172.21.15.187:3300/0 conn(0x7f21680add70 msgr2=0x7f21680b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 --2- 172.21.15.112:0/4045026016 >> v2:172.21.15.187:3300/0 conn(0x7f21680add70 0x7f21680b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21680afa00 con 0x7f216806ca10 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f215c0016a0 con 0x7f216806ca10 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 >> v1:172.21.15.112:6789/0 conn(0x7f216806ca10 legacy=0x7f21681477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f216806c630 0x7f2150005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2150006150 0x7f2150006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006f60 0x7f21500073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21680b52f0 con 0x7f2150006150 2024-04-25T00:56:24.834 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f215000dc40 con 0x7f2150006f60 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f215000de40 con 0x7f216806c630 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216d3d7640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2150006150 0x7f2150006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216dbd8640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f216806c630 0x7f2150005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216e3d9640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006f60 0x7f21500073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216d3d7640 1 -- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f216806c630 msgr2=0x7f2150005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216d3d7640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f216806c630 0x7f2150005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216d3d7640 1 -- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006f60 msgr2=0x7f21500073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216d3d7640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006f60 0x7f21500073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.835 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.830+0000 7f216d3d7640 1 -- 172.21.15.112:0/4045026016 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2150005740 con 0x7f2150006150 2024-04-25T00:56:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216d3d7640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2150006150 0x7f2150006a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f215c00c210 tx=0x7f215c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7d4e23f1365670 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c007d70 con 0x7f2150006150 2024-04-25T00:56:24.836 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215c0a9020 con 0x7f2150006150 2024-04-25T00:56:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216cbd6640 1 -- 172.21.15.112:0/4045026016 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c005ae0 con 0x7f2150006150 2024-04-25T00:56:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- 172.21.15.112:0/4045026016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2150006150 msgr2=0x7f2150006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 --2- 172.21.15.112:0/4045026016 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2150006150 0x7f2150006a20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f215c00c210 tx=0x7f215c01cca0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216e3d9640 1 -- 172.21.15.112:0/4045026016 reap_dead start 2024-04-25T00:56:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- 172.21.15.112:0/4045026016 shutdown_connections 2024-04-25T00:56:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- 172.21.15.112:0/4045026016 >> 172.21.15.112:0/4045026016 conn(0x7f21680b5980 msgr2=0x7f21680b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- 172.21.15.112:0/4045026016 shutdown_connections 2024-04-25T00:56:24.837 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- 172.21.15.112:0/4045026016 wait complete. 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 Processor -- start 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- start start 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006150 0x7f21681439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216806f650 0x7f21681e0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2168071700 0x7f2168071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f21680b3810 con 0x7f216806f650 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f21680b3690 con 0x7f2150006150 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216fe3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f21680b3990 con 0x7f2168071700 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216d3d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216806f650 0x7f21681e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216d3d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216806f650 0x7f21681e0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57836/0 (socket says 172.21.15.112:57836) 2024-04-25T00:56:24.840 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216d3d7640 1 -- 172.21.15.112:0/2329054133 learned_addr learned my addr 172.21.15.112:0/2329054133 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.834+0000 7f216dbd8640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006150 0x7f21681439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216e3d9640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2168071700 0x7f2168071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216d3d7640 1 -- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2168071700 msgr2=0x7f2168071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216d3d7640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2168071700 0x7f2168071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216d3d7640 1 -- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006150 msgr2=0x7f21681439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216d3d7640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006150 0x7f21681439b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216d3d7640 1 -- 172.21.15.112:0/2329054133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216806fdf0 con 0x7f216806f650 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216e3d9640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2168071700 0x7f2168071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216d3d7640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216806f650 0x7f21681e0100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f215c0ab040 tx=0x7f2150007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f21667fc640 1 -- 172.21.15.112:0/2329054133 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c004c00 con 0x7f216806f650 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f21667fc640 1 -- 172.21.15.112:0/2329054133 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215c004da0 con 0x7f216806f650 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21681d1510 con 0x7f216806f650 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f21667fc640 1 -- 172.21.15.112:0/2329054133 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c0a59a0 con 0x7f216806f650 2024-04-25T00:56:24.841 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2168142870 con 0x7f216806f650 2024-04-25T00:56:24.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216804f580 con 0x7f216806f650 2024-04-25T00:56:24.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f21667fc640 1 -- 172.21.15.112:0/2329054133 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f215c0a5b40 con 0x7f216806f650 2024-04-25T00:56:24.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f21667fc640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f21540678f0 0x7f2154069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:24.842 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f21667fc640 1 -- 172.21.15.112:0/2329054133 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f215c11f6f0 con 0x7f216806f650 2024-04-25T00:56:24.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216dbd8640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f21540678f0 0x7f2154069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:24.843 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.838+0000 7f216dbd8640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f21540678f0 0x7f2154069db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2158007440 tx=0x7f215800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:24.845 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:24.842+0000 7f21667fc640 1 -- 172.21.15.112:0/2329054133 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f216806fdf0 con 0x7f216806f650 2024-04-25T00:56:25.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.026+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f216806d090 con 0x7f216806f650 2024-04-25T00:56:25.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.026+0000 7f21667fc640 1 -- 172.21.15.112:0/2329054133 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f215c0169b0 con 0x7f216806f650 2024-04-25T00:56:25.029 INFO:teuthology.orchestra.run.smithi112.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:56:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f21540678f0 msgr2=0x7f2154069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f21540678f0 0x7f2154069db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2158007440 tx=0x7f215800b040 comp rx=0 tx=0).stop 2024-04-25T00:56:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216806f650 msgr2=0x7f21681e0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216806f650 0x7f21681e0100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f215c0ab040 tx=0x7f2150007a50 comp rx=0 tx=0).stop 2024-04-25T00:56:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 shutdown_connections 2024-04-25T00:56:25.031 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f2168071700 0x7f2168071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f216806f650 0x7f21681e0100 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f21540678f0 0x7f2154069db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 --2- 172.21.15.112:0/2329054133 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2150006150 0x7f21681439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 >> 172.21.15.112:0/2329054133 conn(0x7f21680b5980 msgr2=0x7f21680b5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 shutdown_connections 2024-04-25T00:56:25.032 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25.030+0000 7f216fe3c640 1 -- 172.21.15.112:0/2329054133 wait complete. 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:56:25.046 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:56:25.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:25.047 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2024-04-25T00:56:25.052 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:56:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:56:25.052 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T00:56:25.053 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42483sec preferred_lft 42483sec 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 02:42:63:f8:7a:65 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:56:25.054 INFO:teuthology.orchestra.run.smithi112.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T00:56:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:56:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T00:56:25.055 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:56:25.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:25.055 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:56:25.102 INFO:teuthology.orchestra.run.smithi112.stdout:1 2024-04-25T00:56:25.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:25.104 DEBUG:teuthology.orchestra.run.smithi112:> route 2024-04-25T00:56:25.154 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2024-04-25T00:56:25.154 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:56:25.154 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T00:56:25.154 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:56:25.155 INFO:teuthology.orchestra.run.smithi112.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T00:56:25.155 INFO:teuthology.orchestra.run.smithi112.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T00:56:25.155 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:56:25.155 INFO:teuthology.orchestra.run.smithi112.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:56:25.155 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:56:25.155 INFO:teuthology.orchestra.run.smithi112.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T00:56:25.155 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:56:25.156 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:56:25.156 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2024-04-25T00:56:25.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:56:25.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:56:25.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:56:25.156 DEBUG:teuthology.orchestra.run.smithi112:> ') 2024-04-25T00:56:25.241 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:56:25.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:56:25.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:25.276 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T00:56:25.322 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:56:25.323 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:56:25.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:25.323 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2024-04-25T00:56:25.377 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T00:56:25.377 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:56:25.377 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:25.423 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:56:25.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:56:25.423 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:25.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-25T00:56:25.533 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:56:25.580 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-25T00:56:25.580 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-25T00:56:25.580 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-25T00:56:25.580 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-25T00:56:25.580 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-25T00:56:25.580 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/run/lock 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/fs/nfsd 2024-04-25T00:56:25.581 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/run/rpc_pipefs 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:56:25.582 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:56:25.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:25.583 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-25T00:56:25.635 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:56:25.635 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T00:56:25.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-25T00:56:25.694 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:56:25.743 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-25T00:56:25.743 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/run/lock 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-25T00:56:25.744 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/fs/nfsd 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/run/rpc_pipefs 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-25T00:56:25.745 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:56:25.746 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:56:25.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:25.747 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-25T00:56:25.755 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:2024-04-25T00:56:25.754+0000 7f012161f580 -1 init, newargv = 0x564e5e956490 newargc=15 2024-04-25T00:56:25.755 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[36843]: starting ceph client 2024-04-25T00:56:25.760 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[36843]: starting fuse 2024-04-25T00:56:26.861 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2024-04-25T00:56:26.877 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:56:26.930 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2024-04-25T00:56:26.930 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2024-04-25T00:56:26.930 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2024-04-25T00:56:26.930 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/run/lock 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2024-04-25T00:56:26.931 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/fs/nfsd 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/run/rpc_pipefs 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2024-04-25T00:56:26.932 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:56:26.933 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:56:26.933 INFO:teuthology.orchestra.run.smithi112.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:26.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:26.934 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2024-04-25T00:56:26.979 INFO:teuthology.orchestra.run.smithi112.stdout:45 2024-04-25T00:56:26.979 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T00:56:26.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:56:26.979 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> import re 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> import os 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T00:56:26.980 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> return f 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.0")) 2024-04-25T00:56:26.981 DEBUG:teuthology.orchestra.run.smithi112:> ' 2024-04-25T00:56:27.105 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.0.36843.asok 2024-04-25T00:56:27.108 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:56:27.116 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36843.asok 2024-04-25T00:56:27.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:27.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36843.asok status 2024-04-25T00:56:27.273 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:27.273 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2024-04-25T00:56:27.273 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:56:27.273 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:56:27.273 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "0", 2024-04-25T00:56:27.273 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2024-04-25T00:56:27.273 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T00:56:27.273 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "36843", 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 5115, 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 5115 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:56:27.274 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3776316638 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: } 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 3776316638 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.5115 192.168.0.1:0/3776316638", 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/3776316638", 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 16, 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 29, 2024-04-25T00:56:27.275 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:56:27.276 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2024-04-25T00:56:27.276 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2024-04-25T00:56:27.276 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:27.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:27.287 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2024-04-25T00:56:27.288 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:27.293 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2024-04-25T00:56:27.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:27.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:27.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:27.347 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-04-25T00:56:27.395 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.JoNoi06Tsk 2024-04-25T00:56:27.395 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:56:27.395 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/tmp/tmp.JoNoi06Tsk 2024-04-25T00:56:27.443 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p dir1/dir2/dir3') 2024-04-25T00:56:27.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:56:27.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:56:27.576 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 755 /tmp/test-cephfsgw7smiww') 2024-04-25T00:56:27.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:56:27.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:56:27.630 INFO:tasks.cephfs.test_cephfs_shell:Running script "/tmp/test-cephfsgw7smiww" 2024-04-25T00:56:27.630 DEBUG:teuthology.orchestra.run.smithi112:> cephfs-shell -b /tmp/test-cephfsgw7smiww 2024-04-25T00:56:27.987 INFO:teuthology.orchestra.run.smithi112.stderr:Unrecognized command 2024-04-25T00:56:28.016 DEBUG:teuthology.orchestra.run:got remote process result: 127 2024-04-25T00:56:28.017 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T00:56:28.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:56:28.056 DEBUG:teuthology.orchestra.run.smithi187:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T00:56:28.095 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T00:56:28.095 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T00:56:28.095 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:56:28.095 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:56:28.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:28.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.266+0000 7f8720ee8640 1 Processor -- start 2024-04-25T00:56:28.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.266+0000 7f8720ee8640 1 -- start start 2024-04-25T00:56:28.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8720ee8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c0722d0 0x7f871c072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8720ee8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f871c09c270 0x7f871c147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8720ee8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871c0bbec0 con 0x7f871c147650 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8720ee8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f871c0baaf0 con 0x7f871c09c270 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8720ee8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f871c073100 con 0x7f871c0722d0 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8713fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f871c09c270 0x7f871c147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8713fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f871c09c270 0x7f871c147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57872/0 (socket says 172.21.15.112:57872) 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8713fff640 1 -- 172.21.15.112:0/973740571 learned_addr learned my addr 172.21.15.112:0/973740571 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f871a59c640 1 --2- 172.21.15.112:0/973740571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c0722d0 0x7f871c072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8713fff640 1 -- 172.21.15.112:0/973740571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c0722d0 msgr2=0x7f871c072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8713fff640 1 --2- 172.21.15.112:0/973740571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c0722d0 0x7f871c072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8713fff640 1 -- 172.21.15.112:0/973740571 >> v1:172.21.15.187:6789/0 conn(0x7f871c147650 legacy=0x7f871c14ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:28.272 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8713fff640 1 -- 172.21.15.112:0/973740571 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c0bc5c0 con 0x7f871c09c270 2024-04-25T00:56:28.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8713fff640 1 --2- 172.21.15.112:0/973740571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c09c270 0x7f871c147d00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8704009870 tx=0x7f870401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2d34f11b2aabfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8719d9b640 1 -- 172.21.15.112:0/973740571 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87040aa610 con 0x7f871c09c270 2024-04-25T00:56:28.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8719d9b640 1 -- 172.21.15.112:0/973740571 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87040af070 con 0x7f871c09c270 2024-04-25T00:56:28.274 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.270+0000 7f8719d9b640 1 -- 172.21.15.112:0/973740571 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87040aa9d0 con 0x7f871c09c270 2024-04-25T00:56:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- 172.21.15.112:0/973740571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c09c270 msgr2=0x7f871c147d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 --2- 172.21.15.112:0/973740571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c09c270 0x7f871c147d00 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8704009870 tx=0x7f870401c4f0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- 172.21.15.112:0/973740571 shutdown_connections 2024-04-25T00:56:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 --2- 172.21.15.112:0/973740571 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c09c270 0x7f871c147d00 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 --2- 172.21.15.112:0/973740571 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c0722d0 0x7f871c072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- 172.21.15.112:0/973740571 >> 172.21.15.112:0/973740571 conn(0x7f871c09e8c0 msgr2=0x7f871c09ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- 172.21.15.112:0/973740571 shutdown_connections 2024-04-25T00:56:28.275 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- 172.21.15.112:0/973740571 wait complete. 2024-04-25T00:56:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 Processor -- start 2024-04-25T00:56:28.276 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- start start 2024-04-25T00:56:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f871c0722d0 0x7f871c14c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c09c270 0x7f871c14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c147650 0x7f871c0b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f871c154f10 con 0x7f871c147650 2024-04-25T00:56:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f871c09ac50 con 0x7f871c0722d0 2024-04-25T00:56:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f871c155090 con 0x7f871c09c270 2024-04-25T00:56:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c147650 0x7f871c0b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.278 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c147650 0x7f871c0b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57880/0 (socket says 172.21.15.112:57880) 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 -- 172.21.15.112:0/2805378407 learned_addr learned my addr 172.21.15.112:0/2805378407 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8713fff640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c09c270 0x7f871c14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871a59c640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f871c0722d0 0x7f871c14c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 -- 172.21.15.112:0/2805378407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c09c270 msgr2=0x7f871c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c09c270 0x7f871c14dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 -- 172.21.15.112:0/2805378407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f871c0722d0 msgr2=0x7f871c14c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f871c0722d0 0x7f871c14c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 -- 172.21.15.112:0/2805378407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c14d570 con 0x7f871c147650 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f871ad9d640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c147650 0x7f871c0b0a50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f871400c7b0 tx=0x7f871400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8712ffd640 1 -- 172.21.15.112:0/2805378407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714016020 con 0x7f871c147650 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8712ffd640 1 -- 172.21.15.112:0/2805378407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871400a9f0 con 0x7f871c147650 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c0b3670 con 0x7f871c147650 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8712ffd640 1 -- 172.21.15.112:0/2805378407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871400ade0 con 0x7f871c147650 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c14e2c0 con 0x7f871c147650 2024-04-25T00:56:28.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.274+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c14beb0 con 0x7f871c147650 2024-04-25T00:56:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.278+0000 7f8712ffd640 1 -- 172.21.15.112:0/2805378407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8714019020 con 0x7f871c147650 2024-04-25T00:56:28.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.278+0000 7f8712ffd640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f86fc0678c0 0x7f86fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.278+0000 7f8712ffd640 1 -- 172.21.15.112:0/2805378407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f87140858b0 con 0x7f871c147650 2024-04-25T00:56:28.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.278+0000 7f871a59c640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f86fc0678c0 0x7f86fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.278+0000 7f871a59c640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f86fc0678c0 0x7f86fc069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f870c0096f0 tx=0x7f870c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.282+0000 7f8712ffd640 1 -- 172.21.15.112:0/2805378407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f871c14beb0 con 0x7f871c147650 2024-04-25T00:56:28.451 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.446+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f871c04f580 con 0x7f871c147650 2024-04-25T00:56:28.453 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.450+0000 7f8712ffd640 1 -- 172.21.15.112:0/2805378407 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16717 (secure 0 0 0) 0x7f8714020070 con 0x7f871c147650 2024-04-25T00:56:28.454 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:28.454 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:20.226650+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:56:16.783920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:56:17.569481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f86fc0678c0 msgr2=0x7f86fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f86fc0678c0 0x7f86fc069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f870c0096f0 tx=0x7f870c008040 comp rx=0 tx=0).stop 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c147650 msgr2=0x7f871c0b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c147650 0x7f871c0b0a50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f871400c7b0 tx=0x7f871400cc70 comp rx=0 tx=0).stop 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 shutdown_connections 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f871c147650 0x7f871c0b0a50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f871c09c270 0x7f871c14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f86fc0678c0 0x7f86fc069d80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 --2- 172.21.15.112:0/2805378407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f871c0722d0 0x7f871c14c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 >> 172.21.15.112:0/2805378407 conn(0x7f871c09e8c0 msgr2=0x7f871c0b8560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 shutdown_connections 2024-04-25T00:56:28.456 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.454+0000 7f8720ee8640 1 -- 172.21.15.112:0/2805378407 wait complete. 2024-04-25T00:56:28.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:56:16.783920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:56:17.569481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:56:28.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:56:28.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f896b640 1 Processor -- start 2024-04-25T00:56:28.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f896b640 1 -- start start 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f896b640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f29f40affb0 0x7f29f409da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f896b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29f409dfa0 0x7f29f4148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f896b640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29f40b1390 con 0x7f29f40b0390 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f896b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f29f40b1d70 con 0x7f29f409dfa0 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f896b640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f29f40b1600 con 0x7f29f40affb0 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f1d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29f409dfa0 0x7f29f4148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f1d9b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f29f409dfa0 0x7f29f4148800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57886/0 (socket says 172.21.15.112:57886) 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f1d9b640 1 -- 172.21.15.112:0/2476512954 learned_addr learned my addr 172.21.15.112:0/2476512954 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f1d9b640 1 -- 172.21.15.112:0/2476512954 >> v2:172.21.15.187:3300/0 conn(0x7f29f40affb0 msgr2=0x7f29f409da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f259c640 1 --2- 172.21.15.112:0/2476512954 >> v2:172.21.15.187:3300/0 conn(0x7f29f40affb0 0x7f29f409da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f1d9b640 1 --2- 172.21.15.112:0/2476512954 >> v2:172.21.15.187:3300/0 conn(0x7f29f40affb0 0x7f29f409da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f1d9b640 1 -- 172.21.15.112:0/2476512954 >> v1:172.21.15.187:6789/0 conn(0x7f29f40b0390 legacy=0x7f29f4145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:28.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f1d9b640 1 -- 172.21.15.112:0/2476512954 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f40b1fa0 con 0x7f29f409dfa0 2024-04-25T00:56:28.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.598+0000 7f29f259c640 1 --2- 172.21.15.112:0/2476512954 >> v2:172.21.15.187:3300/0 conn(0x7f29f40affb0 0x7f29f409da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:28.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f1d9b640 1 --2- 172.21.15.112:0/2476512954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f4148800 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f29f40b6670 tx=0x7f29e801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43607d48da019c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f159a640 1 -- 172.21.15.112:0/2476512954 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80ab650 con 0x7f29f409dfa0 2024-04-25T00:56:28.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f159a640 1 -- 172.21.15.112:0/2476512954 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e80b0070 con 0x7f29f409dfa0 2024-04-25T00:56:28.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f159a640 1 -- 172.21.15.112:0/2476512954 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e80aba10 con 0x7f29f409dfa0 2024-04-25T00:56:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- 172.21.15.112:0/2476512954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 msgr2=0x7f29f4148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 --2- 172.21.15.112:0/2476512954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f4148800 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f29f40b6670 tx=0x7f29e801c450 comp rx=0 tx=0).stop 2024-04-25T00:56:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- 172.21.15.112:0/2476512954 shutdown_connections 2024-04-25T00:56:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 --2- 172.21.15.112:0/2476512954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f4148800 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 --2- 172.21.15.112:0/2476512954 >> v2:172.21.15.187:3300/0 conn(0x7f29f40affb0 0x7f29f409da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- 172.21.15.112:0/2476512954 >> 172.21.15.112:0/2476512954 conn(0x7f29f40b8c80 msgr2=0x7f29f40b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- 172.21.15.112:0/2476512954 shutdown_connections 2024-04-25T00:56:28.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- 172.21.15.112:0/2476512954 wait complete. 2024-04-25T00:56:28.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 Processor -- start 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- start start 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f41e18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f40affb0 0x7f29f41e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f40b0390 0x7f29f41de0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f29f4152a90 con 0x7f29f409dfa0 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f29f4149150 con 0x7f29f40b0390 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f29f4152c10 con 0x7f29f40affb0 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f41e18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f41e18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57892/0 (socket says 172.21.15.112:57892) 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 -- 172.21.15.112:0/2133979346 learned_addr learned my addr 172.21.15.112:0/2133979346 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f2d9d640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f40b0390 0x7f29f41de0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f1d9b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f40affb0 0x7f29f41e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.607 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 -- 172.21.15.112:0/2133979346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f40affb0 msgr2=0x7f29f41e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f40affb0 0x7f29f41e1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 -- 172.21.15.112:0/2133979346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f40b0390 msgr2=0x7f29f41de0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f40b0390 0x7f29f41de0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 -- 172.21.15.112:0/2133979346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f4141d30 con 0x7f29f409dfa0 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f259c640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f41e18a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f29e400c7c0 tx=0x7f29e400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29e2ffd640 1 -- 172.21.15.112:0/2133979346 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4007a30 con 0x7f29f409dfa0 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29e2ffd640 1 -- 172.21.15.112:0/2133979346 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29e4007bd0 con 0x7f29f409dfa0 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f409b4b0 con 0x7f29f409dfa0 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29e2ffd640 1 -- 172.21.15.112:0/2133979346 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e40113f0 con 0x7f29f409dfa0 2024-04-25T00:56:28.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.602+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f409b1d0 con 0x7f29f409dfa0 2024-04-25T00:56:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.606+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f40b1040 con 0x7f29f409dfa0 2024-04-25T00:56:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.606+0000 7f29e2ffd640 1 -- 172.21.15.112:0/2133979346 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f29e4011590 con 0x7f29f409dfa0 2024-04-25T00:56:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.606+0000 7f29e2ffd640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29d40678c0 0x7f29d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.606+0000 7f29f1d9b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29d40678c0 0x7f29d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.606+0000 7f29e2ffd640 1 -- 172.21.15.112:0/2133979346 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f29e4085dc0 con 0x7f29f409dfa0 2024-04-25T00:56:28.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.606+0000 7f29f1d9b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29d40678c0 0x7f29d4069d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f29e80af000 tx=0x7f29e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.610+0000 7f29e2ffd640 1 -- 172.21.15.112:0/2133979346 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f29e4015020 con 0x7f29f409dfa0 2024-04-25T00:56:28.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.758+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f29f404f5f0 con 0x7f29f409dfa0 2024-04-25T00:56:28.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.758+0000 7f29e2ffd640 1 -- 172.21.15.112:0/2133979346 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f29e4056990 con 0x7f29f409dfa0 2024-04-25T00:56:28.762 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:56:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29d40678c0 msgr2=0x7f29d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29d40678c0 0x7f29d4069d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f29e80af000 tx=0x7f29e8008040 comp rx=0 tx=0).stop 2024-04-25T00:56:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 msgr2=0x7f29f41e18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f41e18a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f29e400c7c0 tx=0x7f29e400cc80 comp rx=0 tx=0).stop 2024-04-25T00:56:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 shutdown_connections 2024-04-25T00:56:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f29d40678c0 0x7f29d4069d80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f29f40b0390 0x7f29f41de0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f29f40affb0 0x7f29f41e1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 --2- 172.21.15.112:0/2133979346 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f29f409dfa0 0x7f29f41e18a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 >> 172.21.15.112:0/2133979346 conn(0x7f29f40b8c80 msgr2=0x7f29f40b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 shutdown_connections 2024-04-25T00:56:28.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.762+0000 7f29f896b640 1 -- 172.21.15.112:0/2133979346 wait complete. 2024-04-25T00:56:28.779 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:56:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.894+0000 7f179c5b9640 1 Processor -- start 2024-04-25T00:56:28.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.894+0000 7f179c5b9640 1 -- start start 2024-04-25T00:56:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179c5b9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17940affb0 0x7f179409da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179c5b9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f179409dfa0 0x7f1794148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179c5b9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17940b1390 con 0x7f17940b0390 2024-04-25T00:56:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179c5b9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f17940b1d70 con 0x7f179409dfa0 2024-04-25T00:56:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179c5b9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f17940b1600 con 0x7f17940affb0 2024-04-25T00:56:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179ab56640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f17940b0390 0x7f1794145190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58958/0 (socket says 172.21.15.112:58958) 2024-04-25T00:56:28.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179ab56640 1 -- 172.21.15.112:0/3959211479 learned_addr learned my addr 172.21.15.112:0/3959211479 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799b54640 1 --2- 172.21.15.112:0/3959211479 >> v2:172.21.15.187:3300/0 conn(0x7f179409dfa0 0x7f1794148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3993327052 0 0) 0x7f17940b1390 con 0x7f17940b0390 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179a355640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17940affb0 0x7f179409da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1780003530 con 0x7f17940b0390 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 104650523 0 0) 0x7f1780003530 con 0x7f17940b0390 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17940affb0 msgr2=0x7f179409da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17940affb0 0x7f179409da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 >> v2:172.21.15.187:3300/0 conn(0x7f179409dfa0 msgr2=0x7f1794148800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 --2- 172.21.15.112:0/3959211479 >> v2:172.21.15.187:3300/0 conn(0x7f179409dfa0 0x7f1794148800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17940b1fa0 con 0x7f17940b0390 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f178c001a00 con 0x7f17940b0390 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 >> v1:172.21.15.112:6789/0 conn(0x7f17940b0390 legacy=0x7f1794145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17800039f0 0x7f1780003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1780004810 0x7f1780004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17940affb0 0x7f17800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f17940b1600 con 0x7f17800039f0 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f178000fe90 con 0x7f1780004810 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1780010040 con 0x7f17940affb0 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179a355640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17800039f0 0x7f1780003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799b54640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1780004810 0x7f1780004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179a355640 1 -- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17940affb0 msgr2=0x7f17800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179a355640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17940affb0 0x7f17800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179a355640 1 -- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1780004810 msgr2=0x7f1780004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179a355640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1780004810 0x7f1780004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179a355640 1 -- 172.21.15.112:0/3959211479 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1780003530 con 0x7f17800039f0 2024-04-25T00:56:28.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799b54640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1780004810 0x7f1780004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f179a355640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17800039f0 0x7f1780003e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f179000ba00 tx=0x7f179001c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6a134f1c8c07d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179000b600 con 0x7f17800039f0 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17900b1070 con 0x7f17800039f0 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.898+0000 7f1799353640 1 -- 172.21.15.112:0/3959211479 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17900ac860 con 0x7f17800039f0 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- 172.21.15.112:0/3959211479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17800039f0 msgr2=0x7f1780003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 --2- 172.21.15.112:0/3959211479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f17800039f0 0x7f1780003e10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f179000ba00 tx=0x7f179001c620 comp rx=0 tx=0).stop 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 -- 172.21.15.112:0/3959211479 reap_dead start 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- 172.21.15.112:0/3959211479 shutdown_connections 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- 172.21.15.112:0/3959211479 >> 172.21.15.112:0/3959211479 conn(0x7f17940b8c80 msgr2=0x7f17940b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:28.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- 172.21.15.112:0/3959211479 shutdown_connections 2024-04-25T00:56:28.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- 172.21.15.112:0/3959211479 wait complete. 2024-04-25T00:56:28.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 Processor -- start 2024-04-25T00:56:28.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- start start 2024-04-25T00:56:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17800039f0 0x7f17941df360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17941df8a0 0x7f1794143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1794143940 0x7f179413e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1794148ec0 con 0x7f1794143940 2024-04-25T00:56:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1794148d40 con 0x7f17800039f0 2024-04-25T00:56:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1794149040 con 0x7f17941df8a0 2024-04-25T00:56:28.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1794143940 0x7f179413e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1794143940 0x7f179413e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57912/0 (socket says 172.21.15.112:57912) 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 -- 172.21.15.112:0/345326468 learned_addr learned my addr 172.21.15.112:0/345326468 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 -- 172.21.15.112:0/345326468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17941df8a0 msgr2=0x7f1794143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17941df8a0 0x7f1794143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 -- 172.21.15.112:0/345326468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17800039f0 msgr2=0x7f17941df360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17800039f0 0x7f17941df360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 -- 172.21.15.112:0/345326468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1794143dc0 con 0x7f1794143940 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179ab56640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1794143940 0x7f179413e870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f17940b0770 tx=0x7f178c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f178affd640 1 -- 172.21.15.112:0/345326468 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c00ce70 con 0x7f1794143940 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f178affd640 1 -- 172.21.15.112:0/345326468 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c00a610 con 0x7f1794143940 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17941e0260 con 0x7f1794143940 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f178affd640 1 -- 172.21.15.112:0/345326468 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c00aa00 con 0x7f1794143940 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17941dff80 con 0x7f1794143940 2024-04-25T00:56:28.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.902+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1794142b30 con 0x7f1794143940 2024-04-25T00:56:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.906+0000 7f178affd640 1 -- 172.21.15.112:0/345326468 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f178c00aba0 con 0x7f1794143940 2024-04-25T00:56:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.906+0000 7f178affd640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f17780678f0 0x7f1778069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.906+0000 7f179a355640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f17780678f0 0x7f1778069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.906+0000 7f178affd640 1 -- 172.21.15.112:0/345326468 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f178c085920 con 0x7f1794143940 2024-04-25T00:56:28.909 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.906+0000 7f179a355640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f17780678f0 0x7f1778069db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17900b0000 tx=0x7f1790004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:28.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:28.910+0000 7f178affd640 1 -- 172.21.15.112:0/345326468 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1794143dc0 con 0x7f1794143940 2024-04-25T00:56:29.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f17940b1040 con 0x7f1794143940 2024-04-25T00:56:29.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f178affd640 1 -- 172.21.15.112:0/345326468 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f178c01c350 con 0x7f1794143940 2024-04-25T00:56:29.068 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:56:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f17780678f0 msgr2=0x7f1778069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f17780678f0 0x7f1778069db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f17900b0000 tx=0x7f1790004740 comp rx=0 tx=0).stop 2024-04-25T00:56:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1794143940 msgr2=0x7f179413e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1794143940 0x7f179413e870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f17940b0770 tx=0x7f178c00cc20 comp rx=0 tx=0).stop 2024-04-25T00:56:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 shutdown_connections 2024-04-25T00:56:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1794143940 0x7f179413e870 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f17941df8a0 0x7f1794143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f17780678f0 0x7f1778069db0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 --2- 172.21.15.112:0/345326468 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f17800039f0 0x7f17941df360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 >> 172.21.15.112:0/345326468 conn(0x7f17940b8c80 msgr2=0x7f17940b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 shutdown_connections 2024-04-25T00:56:29.071 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.066+0000 7f179c5b9640 1 -- 172.21.15.112:0/345326468 wait complete. 2024-04-25T00:56:29.085 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:56:29.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe607c66640 1 Processor -- start 2024-04-25T00:56:29.207 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe607c66640 1 -- start start 2024-04-25T00:56:29.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe607c66640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe60006c170 0x7fe6000aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.208 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe607c66640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6000b34b0 0x7fe6000b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe607c66640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe6000b3e50 0x7fe600147900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe607c66640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fe6000ae8a0 con 0x7fe60006c170 2024-04-25T00:56:29.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe607c66640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe6000b42e0 con 0x7fe6000b3e50 2024-04-25T00:56:29.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe607c66640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe60006c550 con 0x7fe6000b34b0 2024-04-25T00:56:29.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605a02640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe60006c170 0x7fe6000aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.209 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605a02640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe60006c170 0x7fe6000aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57922/0 (socket says 172.21.15.112:57922) 2024-04-25T00:56:29.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605a02640 1 -- 172.21.15.112:0/2422787948 learned_addr learned my addr 172.21.15.112:0/2422787948 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:29.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe606203640 1 --2- 172.21.15.112:0/2422787948 >> v2:172.21.15.187:3300/0 conn(0x7fe6000b3e50 0x7fe600147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605201640 1 --2- 172.21.15.112:0/2422787948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6000b34b0 0x7fe6000b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605a02640 1 -- 172.21.15.112:0/2422787948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6000b34b0 msgr2=0x7fe6000b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:29.210 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605a02640 1 --2- 172.21.15.112:0/2422787948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6000b34b0 0x7fe6000b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605a02640 1 -- 172.21.15.112:0/2422787948 >> v2:172.21.15.187:3300/0 conn(0x7fe6000b3e50 msgr2=0x7fe600147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:29.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605a02640 1 --2- 172.21.15.112:0/2422787948 >> v2:172.21.15.187:3300/0 conn(0x7fe6000b3e50 0x7fe600147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.206+0000 7fe605a02640 1 -- 172.21.15.112:0/2422787948 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6000ae220 con 0x7fe60006c170 2024-04-25T00:56:29.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe605a02640 1 --2- 172.21.15.112:0/2422787948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60006c170 0x7fe6000aedd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009870 tx=0x7fe5fc01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be6b5b4bfac9401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.212 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe604a00640 1 -- 172.21.15.112:0/2422787948 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0b0070 con 0x7fe60006c170 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe604a00640 1 -- 172.21.15.112:0/2422787948 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5fc0ab600 con 0x7fe60006c170 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe604a00640 1 -- 172.21.15.112:0/2422787948 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc0ab9d0 con 0x7fe60006c170 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 -- 172.21.15.112:0/2422787948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60006c170 msgr2=0x7fe6000aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 --2- 172.21.15.112:0/2422787948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60006c170 0x7fe6000aedd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009870 tx=0x7fe5fc01c1d0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 -- 172.21.15.112:0/2422787948 shutdown_connections 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 --2- 172.21.15.112:0/2422787948 >> v2:172.21.15.187:3300/0 conn(0x7fe6000b3e50 0x7fe600147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 --2- 172.21.15.112:0/2422787948 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe6000b34b0 0x7fe6000b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 --2- 172.21.15.112:0/2422787948 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe60006c170 0x7fe6000aedd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 -- 172.21.15.112:0/2422787948 >> 172.21.15.112:0/2422787948 conn(0x7fe6000b8c80 msgr2=0x7fe6000b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 -- 172.21.15.112:0/2422787948 shutdown_connections 2024-04-25T00:56:29.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 -- 172.21.15.112:0/2422787948 wait complete. 2024-04-25T00:56:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 Processor -- start 2024-04-25T00:56:29.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.210+0000 7fe607c66640 1 -- start start 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe60006c170 0x7fe60006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6000b34b0 0x7fe60009bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6000b3e50 0x7fe60009d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe6000b2ea0 con 0x7fe6000b34b0 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe6000adef0 con 0x7fe6000b3e50 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe6000b51f0 con 0x7fe60006c170 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6000b34b0 0x7fe60009bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6000b34b0 0x7fe60009bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57926/0 (socket says 172.21.15.112:57926) 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 -- 172.21.15.112:0/1127754219 learned_addr learned my addr 172.21.15.112:0/1127754219 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605a02640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe60006c170 0x7fe60006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe606203640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6000b3e50 0x7fe60009d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 -- 172.21.15.112:0/1127754219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe60006c170 msgr2=0x7fe60006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe60006c170 0x7fe60006ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 -- 172.21.15.112:0/1127754219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6000b3e50 msgr2=0x7fe60009d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6000b3e50 0x7fe60009d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 -- 172.21.15.112:0/1127754219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60013dad0 con 0x7fe6000b34b0 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605201640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6000b34b0 0x7fe60009bd60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d800 tx=0x7fe5f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe5f67fc640 1 -- 172.21.15.112:0/1127754219 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000b660 con 0x7fe6000b34b0 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe5f67fc640 1 -- 172.21.15.112:0/1127754219 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5f0019070 con 0x7fe6000b34b0 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60009c800 con 0x7fe6000b34b0 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe5f67fc640 1 -- 172.21.15.112:0/1127754219 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000ba00 con 0x7fe6000b34b0 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe600143930 con 0x7fe6000b34b0 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60004f5f0 con 0x7fe6000b34b0 2024-04-25T00:56:29.218 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe5f67fc640 1 -- 172.21.15.112:0/1127754219 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe5f000bba0 con 0x7fe6000b34b0 2024-04-25T00:56:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe5f67fc640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5e40678c0 0x7fe5e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe5f67fc640 1 -- 172.21.15.112:0/1127754219 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fe5f0086050 con 0x7fe6000b34b0 2024-04-25T00:56:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605a02640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5e40678c0 0x7fe5e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.214+0000 7fe605a02640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5e40678c0 0x7fe5e4069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0ad000 tx=0x7fe5fc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.218+0000 7fe5f67fc640 1 -- 172.21.15.112:0/1127754219 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe60013dad0 con 0x7fe6000b34b0 2024-04-25T00:56:29.376 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.374+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe6000b84c0 con 0x7fe6000b34b0 2024-04-25T00:56:29.377 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.374+0000 7fe5f67fc640 1 -- 172.21.15.112:0/1127754219 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fe5f0056c20 con 0x7fe6000b34b0 2024-04-25T00:56:29.377 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:56:29.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5e40678c0 msgr2=0x7fe5e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5e40678c0 0x7fe5e4069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0ad000 tx=0x7fe5fc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:56:29.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6000b34b0 msgr2=0x7fe60009bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6000b34b0 0x7fe60009bd60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000d800 tx=0x7fe5f000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 shutdown_connections 2024-04-25T00:56:29.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe5e40678c0 0x7fe5e4069d80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe6000b3e50 0x7fe60009d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6000b34b0 0x7fe60009bd60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 --2- 172.21.15.112:0/1127754219 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe60006c170 0x7fe60006ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 >> 172.21.15.112:0/1127754219 conn(0x7fe6000b8c80 msgr2=0x7fe600144700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:29.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 shutdown_connections 2024-04-25T00:56:29.380 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.378+0000 7fe607c66640 1 -- 172.21.15.112:0/1127754219 wait complete. 2024-04-25T00:56:29.394 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:56:29.503 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84a282640 1 Processor -- start 2024-04-25T00:56:29.504 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84a282640 1 -- start start 2024-04-25T00:56:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84a282640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 0x7fa8440add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84a282640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa844145200 0x7fa8440b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84a282640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa844072760 0x7fa844072bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84a282640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa8440aeec0 con 0x7fa844145200 2024-04-25T00:56:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84a282640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa8440af780 con 0x7fa844072760 2024-04-25T00:56:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84a282640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa8440b27d0 con 0x7fa8440b23f0 2024-04-25T00:56:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa8437fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa844145200 0x7fa8440b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.505 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa8437fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa844145200 0x7fa8440b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57930/0 (socket says 172.21.15.112:57930) 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa8437fe640 1 -- 172.21.15.112:0/3754971842 learned_addr learned my addr 172.21.15.112:0/3754971842 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84881f640 1 --2- 172.21.15.112:0/3754971842 >> v2:172.21.15.187:3300/0 conn(0x7fa844072760 0x7fa844072bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa8437fe640 1 -- 172.21.15.112:0/3754971842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 msgr2=0x7fa8440add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa843fff640 1 --2- 172.21.15.112:0/3754971842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 0x7fa8440add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa8437fe640 1 --2- 172.21.15.112:0/3754971842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 0x7fa8440add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa8437fe640 1 -- 172.21.15.112:0/3754971842 >> v2:172.21.15.187:3300/0 conn(0x7fa844072760 msgr2=0x7fa844072bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa8437fe640 1 --2- 172.21.15.112:0/3754971842 >> v2:172.21.15.187:3300/0 conn(0x7fa844072760 0x7fa844072bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa8437fe640 1 -- 172.21.15.112:0/3754971842 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8440afa20 con 0x7fa844145200 2024-04-25T00:56:29.506 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.502+0000 7fa84881f640 1 --2- 172.21.15.112:0/3754971842 >> v2:172.21.15.187:3300/0 conn(0x7fa844072760 0x7fa844072bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa8437fe640 1 --2- 172.21.15.112:0/3754971842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844145200 0x7fa8440b3ce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa84406b8a0 tx=0x7fa82c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51791cf086d59b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa842ffd640 1 -- 172.21.15.112:0/3754971842 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0aa660 con 0x7fa844145200 2024-04-25T00:56:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa842ffd640 1 -- 172.21.15.112:0/3754971842 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82c0af070 con 0x7fa844145200 2024-04-25T00:56:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa842ffd640 1 -- 172.21.15.112:0/3754971842 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0aaa20 con 0x7fa844145200 2024-04-25T00:56:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- 172.21.15.112:0/3754971842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844145200 msgr2=0x7fa8440b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 --2- 172.21.15.112:0/3754971842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844145200 0x7fa8440b3ce0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa84406b8a0 tx=0x7fa82c01c520 comp rx=0 tx=0).stop 2024-04-25T00:56:29.508 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- 172.21.15.112:0/3754971842 shutdown_connections 2024-04-25T00:56:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 --2- 172.21.15.112:0/3754971842 >> v2:172.21.15.187:3300/0 conn(0x7fa844072760 0x7fa844072bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 --2- 172.21.15.112:0/3754971842 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844145200 0x7fa8440b3ce0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 --2- 172.21.15.112:0/3754971842 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 0x7fa8440add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- 172.21.15.112:0/3754971842 >> 172.21.15.112:0/3754971842 conn(0x7fa8440b5980 msgr2=0x7fa8440b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- 172.21.15.112:0/3754971842 shutdown_connections 2024-04-25T00:56:29.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- 172.21.15.112:0/3754971842 wait complete. 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 Processor -- start 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- start start 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844072760 0x7fa844147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 0x7fa8441dda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa844145200 0x7fa8441ddf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa84406ce60 con 0x7fa844072760 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa84406cce0 con 0x7fa844145200 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa84406cfe0 con 0x7fa8440b23f0 2024-04-25T00:56:29.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844072760 0x7fa844147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844072760 0x7fa844147ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57932/0 (socket says 172.21.15.112:57932) 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 -- 172.21.15.112:0/1995469496 learned_addr learned my addr 172.21.15.112:0/1995469496 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84881f640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa844145200 0x7fa8441ddf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa8437fe640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 0x7fa8441dda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 -- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 msgr2=0x7fa8441dda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 0x7fa8441dda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 -- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa844145200 msgr2=0x7fa8441ddf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa844145200 0x7fa8441ddf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 -- 172.21.15.112:0/1995469496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84413fd10 con 0x7fa844072760 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84881f640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa844145200 0x7fa8441ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa843fff640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844072760 0x7fa844147ff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa83400ce10 tx=0x7fa83400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa840ff9640 1 -- 172.21.15.112:0/1995469496 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834003ba0 con 0x7fa844072760 2024-04-25T00:56:29.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa840ff9640 1 -- 172.21.15.112:0/1995469496 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa834003d40 con 0x7fa844072760 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa844071b20 con 0x7fa844072760 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa840ff9640 1 -- 172.21.15.112:0/1995469496 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834004240 con 0x7fa844072760 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.506+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa844149390 con 0x7fa844072760 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.510+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa844146f80 con 0x7fa844072760 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.510+0000 7fa840ff9640 1 -- 172.21.15.112:0/1995469496 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa8340043e0 con 0x7fa844072760 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.510+0000 7fa840ff9640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa8240678c0 0x7fa824069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.510+0000 7fa840ff9640 1 -- 172.21.15.112:0/1995469496 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fa834086780 con 0x7fa844072760 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.510+0000 7fa8437fe640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa8240678c0 0x7fa824069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.510+0000 7fa8437fe640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa8240678c0 0x7fa824069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0ae000 tx=0x7fa82c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.514+0000 7fa840ff9640 1 -- 172.21.15.112:0/1995469496 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa834057350 con 0x7fa844072760 2024-04-25T00:56:29.677 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.674+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa84404f580 con 0x7fa844072760 2024-04-25T00:56:29.678 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.674+0000 7fa840ff9640 1 -- 172.21.15.112:0/1995469496 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fa834056c50 con 0x7fa844072760 2024-04-25T00:56:29.678 INFO:teuthology.orchestra.run.smithi112.stderr:listed 0 entries 2024-04-25T00:56:29.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa8240678c0 msgr2=0x7fa824069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa8240678c0 0x7fa824069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0ae000 tx=0x7fa82c0a7000 comp rx=0 tx=0).stop 2024-04-25T00:56:29.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844072760 msgr2=0x7fa844147ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844072760 0x7fa844147ff0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa83400ce10 tx=0x7fa83400a660 comp rx=0 tx=0).stop 2024-04-25T00:56:29.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 shutdown_connections 2024-04-25T00:56:29.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa8240678c0 0x7fa824069d80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.680 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa844145200 0x7fa8441ddf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa8440b23f0 0x7fa8441dda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 --2- 172.21.15.112:0/1995469496 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa844072760 0x7fa844147ff0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 >> 172.21.15.112:0/1995469496 conn(0x7fa8440b5980 msgr2=0x7fa8440b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:29.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 shutdown_connections 2024-04-25T00:56:29.681 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.678+0000 7fa84a282640 1 -- 172.21.15.112:0/1995469496 wait complete. 2024-04-25T00:56:29.695 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2024-04-25T00:56:29.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:29.695 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T00:56:29.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi112.stderr:ceph-fuse[36843]: fuse finished with error 0 and tester_r 0 2024-04-25T00:56:29.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:56:29.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:29.707 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:56:29.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:29.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffab19e2640 1 Processor -- start 2024-04-25T00:56:29.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffab19e2640 1 -- start start 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffab19e2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffaac06c630 0x7ffaac0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffab19e2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaac1485e0 con 0x7ffaac06ca10 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffab19e2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaac0af760 con 0x7ffaac0add70 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffab19e2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ffaac148860 con 0x7ffaac06c630 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaaaffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffaac06c630 0x7ffaac0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaa27fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7ffaac06ca10 0x7ffaac1477c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58968/0 (socket says 172.21.15.112:58968) 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaaaffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ffaac06c630 0x7ffaac0aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57944/0 (socket says 172.21.15.112:57944) 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaa27fc640 1 -- 172.21.15.112:0/2800616850 learned_addr learned my addr 172.21.15.112:0/2800616850 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaaaffd640 1 -- 172.21.15.112:0/2800616850 >> v1:172.21.15.187:6789/0 conn(0x7ffaac0add70 legacy=0x7ffaac147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:29.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaaaffd640 1 -- 172.21.15.112:0/2800616850 >> v1:172.21.15.112:6789/0 conn(0x7ffaac06ca10 legacy=0x7ffaac1477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaaaffd640 1 -- 172.21.15.112:0/2800616850 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac0afa00 con 0x7ffaac06c630 2024-04-25T00:56:29.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaaa7fc640 1 -- 172.21.15.112:0/2800616850 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3899478164 0 0) 0x7ffaac1485e0 con 0x7ffaac06ca10 2024-04-25T00:56:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.954+0000 7ffaaaffd640 1 --2- 172.21.15.112:0/2800616850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac06c630 0x7ffaac0aff80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0097c0 tx=0x7ffa9c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d540f9ade5c5ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaaa7fc640 1 -- 172.21.15.112:0/2800616850 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0b0070 con 0x7ffaac06c630 2024-04-25T00:56:29.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaaa7fc640 1 -- 172.21.15.112:0/2800616850 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9c0ab600 con 0x7ffaac06c630 2024-04-25T00:56:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaaa7fc640 1 -- 172.21.15.112:0/2800616850 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0ab9d0 con 0x7ffaac06c630 2024-04-25T00:56:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- 172.21.15.112:0/2800616850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac06c630 msgr2=0x7ffaac0aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 --2- 172.21.15.112:0/2800616850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac06c630 0x7ffaac0aff80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0097c0 tx=0x7ffa9c01c340 comp rx=0 tx=0).stop 2024-04-25T00:56:29.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- 172.21.15.112:0/2800616850 shutdown_connections 2024-04-25T00:56:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 --2- 172.21.15.112:0/2800616850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac06c630 0x7ffaac0aff80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- 172.21.15.112:0/2800616850 >> 172.21.15.112:0/2800616850 conn(0x7ffaac0b5980 msgr2=0x7ffaac0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- 172.21.15.112:0/2800616850 shutdown_connections 2024-04-25T00:56:29.960 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- 172.21.15.112:0/2800616850 wait complete. 2024-04-25T00:56:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 Processor -- start 2024-04-25T00:56:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- start start 2024-04-25T00:56:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaac06c630 0x7ffaac0b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaac06ca10 0x7ffaac1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac0add70 0x7ffaac143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac152a00 con 0x7ffaac0add70 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac152880 con 0x7ffaac06c630 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ffaac152b80 con 0x7ffaac06ca10 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac0add70 0x7ffaac143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac0add70 0x7ffaac143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57946/0 (socket says 172.21.15.112:57946) 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 -- 172.21.15.112:0/2195314727 learned_addr learned my addr 172.21.15.112:0/2195314727 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaaaffd640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaac06c630 0x7ffaac0b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaa27fc640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaac06ca10 0x7ffaac1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 -- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaac06ca10 msgr2=0x7ffaac1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaac06ca10 0x7ffaac1436b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 -- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaac06c630 msgr2=0x7ffaac0b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaac06c630 0x7ffaac0b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 -- 172.21.15.112:0/2195314727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac0b82a0 con 0x7ffaac0add70 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaaaffd640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaac06c630 0x7ffaac0b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:29.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaab7fe640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac0add70 0x7ffaac143bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400b9e0 tx=0x7ffaa400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaa8ff9640 1 -- 172.21.15.112:0/2195314727 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4019070 con 0x7ffaac0add70 2024-04-25T00:56:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaa8ff9640 1 -- 172.21.15.112:0/2195314727 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaa400caa0 con 0x7ffaac0add70 2024-04-25T00:56:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaac09b370 con 0x7ffaac0add70 2024-04-25T00:56:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffaa8ff9640 1 -- 172.21.15.112:0/2195314727 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4014440 con 0x7ffaac0add70 2024-04-25T00:56:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.958+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaac09b930 con 0x7ffaac0add70 2024-04-25T00:56:29.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.962+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaac04f580 con 0x7ffaac0add70 2024-04-25T00:56:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.962+0000 7ffaa8ff9640 1 -- 172.21.15.112:0/2195314727 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffaa401c070 con 0x7ffaac0add70 2024-04-25T00:56:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.962+0000 7ffaa8ff9640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ffa8c0678c0 0x7ffa8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.962+0000 7ffaa8ff9640 1 -- 172.21.15.112:0/2195314727 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7ffaa4085760 con 0x7ffaac0add70 2024-04-25T00:56:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.962+0000 7ffaaaffd640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ffa8c0678c0 0x7ffa8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:29.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.962+0000 7ffaaaffd640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ffa8c0678c0 0x7ffa8c069d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0ad000 tx=0x7ffa9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:29.967 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:29.966+0000 7ffaa8ff9640 1 -- 172.21.15.112:0/2195314727 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffaa4056330 con 0x7ffaac0add70 2024-04-25T00:56:30.146 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.142+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffaac0aed20 con 0x7ffaac0add70 2024-04-25T00:56:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffaa8ff9640 1 -- 172.21.15.112:0/2195314727 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3537 (secure 0 0 0) 0x7ffaa4055c30 con 0x7ffaac0add70 2024-04-25T00:56:30.147 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:30.147 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3898281312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3898281312},{"type":"v1","addr":"172.21.15.187:6833","nonce":3898281312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:56:18.588797+0000","modified":"2024-04-25T00:56:21.588160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3222796217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3222796217},{"type":"v1","addr":"172.21.15.112:6835","nonce":3222796217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-04-25T00:56:30.147 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ffa8c0678c0 msgr2=0x7ffa8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ffa8c0678c0 0x7ffa8c069d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0ad000 tx=0x7ffa9c008040 comp rx=0 tx=0).stop 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac0add70 msgr2=0x7ffaac143bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac0add70 0x7ffaac143bf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffaa400b9e0 tx=0x7ffaa400bea0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 shutdown_connections 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ffaac0add70 0x7ffaac143bf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ffaac06ca10 0x7ffaac1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ffa8c0678c0 0x7ffa8c069d80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.149 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 --2- 172.21.15.112:0/2195314727 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ffaac06c630 0x7ffaac0b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 >> 172.21.15.112:0/2195314727 conn(0x7ffaac0b5980 msgr2=0x7ffaac0b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 shutdown_connections 2024-04-25T00:56:30.150 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.146+0000 7ffab19e2640 1 -- 172.21.15.112:0/2195314727 wait complete. 2024-04-25T00:56:30.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:30.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af8564640 1 Processor -- start 2024-04-25T00:56:30.279 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af8564640 1 -- start start 2024-04-25T00:56:30.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af8564640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6af006cff0 0x7f6af0141350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af8564640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af0072c30 0x7f6af0141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af8564640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af0071930 con 0x7f6af00b7b10 2024-04-25T00:56:30.280 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af8564640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6af00b7ef0 con 0x7f6af006cff0 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af8564640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6af006cb30 con 0x7f6af0072c30 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af5aff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6af006cff0 0x7f6af0141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af5aff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6af006cff0 0x7f6af0141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57960/0 (socket says 172.21.15.112:57960) 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af5aff640 1 -- 172.21.15.112:0/530676169 learned_addr learned my addr 172.21.15.112:0/530676169 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af6b01640 1 --2- 172.21.15.112:0/530676169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af0072c30 0x7f6af0141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af5aff640 1 -- 172.21.15.112:0/530676169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af0072c30 msgr2=0x7f6af0141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af5aff640 1 --2- 172.21.15.112:0/530676169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af0072c30 0x7f6af0141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af5aff640 1 -- 172.21.15.112:0/530676169 >> v1:172.21.15.112:6789/0 conn(0x7f6af00b7b10 legacy=0x7f6af0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af52fe640 1 -- 172.21.15.112:0/530676169 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2453368921 0 0) 0x7f6af0071930 con 0x7f6af00b7b10 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af5aff640 1 -- 172.21.15.112:0/530676169 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af006c330 con 0x7f6af006cff0 2024-04-25T00:56:30.281 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af6b01640 1 --2- 172.21.15.112:0/530676169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af0072c30 0x7f6af0141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:30.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af5aff640 1 --2- 172.21.15.112:0/530676169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0141350 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009870 tx=0x7f6aec01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c037a6b60a68a844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.278+0000 7f6af52fe640 1 -- 172.21.15.112:0/530676169 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0b2070 con 0x7f6af006cff0 2024-04-25T00:56:30.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af52fe640 1 -- 172.21.15.112:0/530676169 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aec0a9070 con 0x7f6af006cff0 2024-04-25T00:56:30.282 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af52fe640 1 -- 172.21.15.112:0/530676169 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec0ad880 con 0x7f6af006cff0 2024-04-25T00:56:30.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- 172.21.15.112:0/530676169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 msgr2=0x7f6af0141350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 --2- 172.21.15.112:0/530676169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0141350 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009870 tx=0x7f6aec01c560 comp rx=0 tx=0).stop 2024-04-25T00:56:30.283 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- 172.21.15.112:0/530676169 shutdown_connections 2024-04-25T00:56:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 --2- 172.21.15.112:0/530676169 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af0072c30 0x7f6af0141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 --2- 172.21.15.112:0/530676169 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0141350 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- 172.21.15.112:0/530676169 >> 172.21.15.112:0/530676169 conn(0x7f6af0067120 msgr2=0x7f6af0067530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- 172.21.15.112:0/530676169 shutdown_connections 2024-04-25T00:56:30.284 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- 172.21.15.112:0/530676169 wait complete. 2024-04-25T00:56:30.286 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 Processor -- start 2024-04-25T00:56:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- start start 2024-04-25T00:56:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0135f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6af0072c30 0x7f6af01e3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af00b7b10 0x7f6af0145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6af0154840 con 0x7f6af006cff0 2024-04-25T00:56:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6af00b8fd0 con 0x7f6af0072c30 2024-04-25T00:56:30.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6af01549c0 con 0x7f6af00b7b10 2024-04-25T00:56:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0135f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57964/0 (socket says 172.21.15.112:57964) 2024-04-25T00:56:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 -- 172.21.15.112:0/2463722661 learned_addr learned my addr 172.21.15.112:0/2463722661 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af5aff640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6af0072c30 0x7f6af01e3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 -- 172.21.15.112:0/2463722661 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af00b7b10 msgr2=0x7f6af0145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af00b7b10 0x7f6af0145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 -- 172.21.15.112:0/2463722661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6af0072c30 msgr2=0x7f6af01e3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6af0072c30 0x7f6af01e3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 -- 172.21.15.112:0/2463722661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af01e3ac0 con 0x7f6af006cff0 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af6300640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0135f60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c500 tx=0x7f6ae000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6ae6ffd640 1 -- 172.21.15.112:0/2463722661 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0017070 con 0x7f6af006cff0 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6ae6ffd640 1 -- 172.21.15.112:0/2463722661 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ae0003600 con 0x7f6af006cff0 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af0147510 con 0x7f6af006cff0 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6ae6ffd640 1 -- 172.21.15.112:0/2463722661 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00039f0 con 0x7f6af006cff0 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af0147ad0 con 0x7f6af006cff0 2024-04-25T00:56:30.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.282+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af004f580 con 0x7f6af006cff0 2024-04-25T00:56:30.290 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.286+0000 7f6ae6ffd640 1 -- 172.21.15.112:0/2463722661 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6ae001a070 con 0x7f6af006cff0 2024-04-25T00:56:30.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.286+0000 7f6ae6ffd640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ad40678c0 0x7f6ad4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.286+0000 7f6af5aff640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ad40678c0 0x7f6ad4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.286+0000 7f6ae6ffd640 1 -- 172.21.15.112:0/2463722661 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f6ae00850a0 con 0x7f6af006cff0 2024-04-25T00:56:30.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.286+0000 7f6af5aff640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ad40678c0 0x7f6ad4069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0af000 tx=0x7f6aec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.290+0000 7f6ae6ffd640 1 -- 172.21.15.112:0/2463722661 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6ae0055d30 con 0x7f6af006cff0 2024-04-25T00:56:30.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.438+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6af0147ca0 con 0x7f6af006cff0 2024-04-25T00:56:30.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6ae6ffd640 1 -- 172.21.15.112:0/2463722661 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16717 (secure 0 0 0) 0x7f6ae0055630 con 0x7f6af006cff0 2024-04-25T00:56:30.443 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:30.443 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:20.226650+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:56:16.783920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:56:17.569481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:30.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ad40678c0 msgr2=0x7f6ad4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ad40678c0 0x7f6ad4069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0af000 tx=0x7f6aec008040 comp rx=0 tx=0).stop 2024-04-25T00:56:30.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 msgr2=0x7f6af0135f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0135f60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c500 tx=0x7f6ae000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 shutdown_connections 2024-04-25T00:56:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6af00b7b10 0x7f6af0145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6ad40678c0 0x7f6ad4069d80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6af0072c30 0x7f6af01e3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 --2- 172.21.15.112:0/2463722661 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6af006cff0 0x7f6af0135f60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 >> 172.21.15.112:0/2463722661 conn(0x7f6af0067120 msgr2=0x7f6af013ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 shutdown_connections 2024-04-25T00:56:30.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.442+0000 7f6af8564640 1 -- 172.21.15.112:0/2463722661 wait complete. 2024-04-25T00:56:30.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:55:27.587648+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:56:16.783920+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:56:17.569481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:56:30.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae87ea6640 1 Processor -- start 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae87ea6640 1 -- start start 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae87ea6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fae8006dc90 0x7fae80072450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae87ea6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 0x7fae8014fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae87ea6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae8006d520 con 0x7fae8006e460 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae87ea6640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fae801487e0 con 0x7fae8006dc90 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae87ea6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fae8013fa30 con 0x7fae80072990 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85c42640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fae8006e460 0x7fae800b57b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:58992/0 (socket says 172.21.15.112:58992) 2024-04-25T00:56:30.593 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85c42640 1 -- 172.21.15.112:0/2339032734 learned_addr learned my addr 172.21.15.112:0/2339032734 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85441640 1 --2- 172.21.15.112:0/2339032734 >> v2:172.21.15.187:3300/0 conn(0x7fae8006dc90 0x7fae80072450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae86443640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 0x7fae8014fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4135841723 0 0) 0x7fae8006d520 con 0x7fae8006e460 2024-04-25T00:56:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae6c003530 con 0x7fae8006e460 2024-04-25T00:56:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1576941529 0 0) 0x7fae6c003530 con 0x7fae8006e460 2024-04-25T00:56:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 msgr2=0x7fae8014fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 0x7fae8014fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.594 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 >> v2:172.21.15.187:3300/0 conn(0x7fae8006dc90 msgr2=0x7fae80072450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 --2- 172.21.15.112:0/2339032734 >> v2:172.21.15.187:3300/0 conn(0x7fae8006dc90 0x7fae80072450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8006d7c0 con 0x7fae8006e460 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7fae78001a20 con 0x7fae8006e460 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 >> v1:172.21.15.112:6789/0 conn(0x7fae8006e460 legacy=0x7fae800b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 0x7fae6c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae6c004d50 0x7fae6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae8013fa30 con 0x7fae6c003f60 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae6c00b9d0 con 0x7fae6c004d50 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fae6c00bbd0 con 0x7fae80072990 2024-04-25T00:56:30.595 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85441640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae86443640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae6c004d50 0x7fae6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85c42640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 0x7fae6c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85441640 1 -- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 msgr2=0x7fae6c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85441640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 0x7fae6c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85441640 1 -- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae6c004d50 msgr2=0x7fae6c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85441640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae6c004d50 0x7fae6c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85441640 1 -- 172.21.15.112:0/2339032734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6c003530 con 0x7fae6c003f60 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.582+0000 7fae85c42640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae80072990 0x7fae6c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85441640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae6c004810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae70009ae0 tx=0x7fae7001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3523f90cc510c94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.596 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700096b0 con 0x7fae6c003f60 2024-04-25T00:56:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae700a6490 con 0x7fae6c003f60 2024-04-25T00:56:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae84c40640 1 -- 172.21.15.112:0/2339032734 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae700a6880 con 0x7fae6c003f60 2024-04-25T00:56:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- 172.21.15.112:0/2339032734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 msgr2=0x7fae6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 --2- 172.21.15.112:0/2339032734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae6c004810 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fae70009ae0 tx=0x7fae7001cab0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae86443640 1 -- 172.21.15.112:0/2339032734 reap_dead start 2024-04-25T00:56:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- 172.21.15.112:0/2339032734 shutdown_connections 2024-04-25T00:56:30.597 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- 172.21.15.112:0/2339032734 >> 172.21.15.112:0/2339032734 conn(0x7fae800b65c0 msgr2=0x7fae800b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- 172.21.15.112:0/2339032734 shutdown_connections 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- 172.21.15.112:0/2339032734 wait complete. 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 Processor -- start 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- start start 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae80147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae8013b840 0x7fae800b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8013bcc0 0x7fae801e0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fae8009f0e0 con 0x7fae6c003f60 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fae8006e840 con 0x7fae8013bcc0 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fae8009f260 con 0x7fae8013b840 2024-04-25T00:56:30.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae80147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae80147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57990/0 (socket says 172.21.15.112:57990) 2024-04-25T00:56:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 -- 172.21.15.112:0/91796828 learned_addr learned my addr 172.21.15.112:0/91796828 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85441640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae8013b840 0x7fae800b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae86443640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8013bcc0 0x7fae801e0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 -- 172.21.15.112:0/91796828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae8013b840 msgr2=0x7fae800b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae8013b840 0x7fae800b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 -- 172.21.15.112:0/91796828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8013bcc0 msgr2=0x7fae801e0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8013bcc0 0x7fae801e0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.600 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 -- 172.21.15.112:0/91796828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae801de140 con 0x7fae6c003f60 2024-04-25T00:56:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae85c42640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae80147420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae7800dfd0 tx=0x7fae7800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae767fc640 1 -- 172.21.15.112:0/91796828 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78019070 con 0x7fae6c003f60 2024-04-25T00:56:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae767fc640 1 -- 172.21.15.112:0/91796828 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae78004d70 con 0x7fae6c003f60 2024-04-25T00:56:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae801dd940 con 0x7fae6c003f60 2024-04-25T00:56:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae767fc640 1 -- 172.21.15.112:0/91796828 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae780072d0 con 0x7fae6c003f60 2024-04-25T00:56:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.586+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae80147f10 con 0x7fae6c003f60 2024-04-25T00:56:30.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.590+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae80074b90 con 0x7fae6c003f60 2024-04-25T00:56:30.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.590+0000 7fae767fc640 1 -- 172.21.15.112:0/91796828 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fae7801c070 con 0x7fae6c003f60 2024-04-25T00:56:30.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.590+0000 7fae767fc640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fae640678c0 0x7fae64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.590+0000 7fae767fc640 1 -- 172.21.15.112:0/91796828 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fae78086270 con 0x7fae6c003f60 2024-04-25T00:56:30.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.590+0000 7fae85441640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fae640678c0 0x7fae64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.594+0000 7fae85441640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fae640678c0 0x7fae64069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae700ad000 tx=0x7fae700046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.594+0000 7fae767fc640 1 -- 172.21.15.112:0/91796828 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fae78023020 con 0x7fae6c003f60 2024-04-25T00:56:30.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.758+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fae8004f5f0 con 0x7fae6c003f60 2024-04-25T00:56:30.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.758+0000 7fae767fc640 1 -- 172.21.15.112:0/91796828 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fae78056ea0 con 0x7fae6c003f60 2024-04-25T00:56:30.761 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 1 2024-04-25T00:56:30.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fae640678c0 msgr2=0x7fae64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fae640678c0 0x7fae64069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fae700ad000 tx=0x7fae700046d0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 msgr2=0x7fae80147420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae80147420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fae7800dfd0 tx=0x7fae7800b4a0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 shutdown_connections 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fae640678c0 0x7fae64069d80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fae8013bcc0 0x7fae801e0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fae8013b840 0x7fae800b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 --2- 172.21.15.112:0/91796828 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fae6c003f60 0x7fae80147420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 >> 172.21.15.112:0/91796828 conn(0x7fae800b65c0 msgr2=0x7fae80145bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 shutdown_connections 2024-04-25T00:56:30.764 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.762+0000 7fae87ea6640 1 -- 172.21.15.112:0/91796828 wait complete. 2024-04-25T00:56:30.780 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:56:30.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.894+0000 7f692cbe5640 1 Processor -- start 2024-04-25T00:56:30.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.894+0000 7f692cbe5640 1 -- start start 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f692809ded0 0x7f69280b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280add40 0x7f69280b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f692809e2b0 con 0x7f692806caf0 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f69280ae790 con 0x7f692809ded0 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f69280aea70 con 0x7f69280add40 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692659c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f692809ded0 0x7f69280b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692659c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f692809ded0 0x7f69280b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:57998/0 (socket says 172.21.15.112:57998) 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f6926d9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f692806caf0 0x7f69280b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59008/0 (socket says 172.21.15.112:59008) 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692659c640 1 -- 172.21.15.112:0/3344034970 learned_addr learned my addr 172.21.15.112:0/3344034970 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f6925d9b640 1 --2- 172.21.15.112:0/3344034970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280add40 0x7f69280b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692659c640 1 -- 172.21.15.112:0/3344034970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280add40 msgr2=0x7f69280b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692659c640 1 --2- 172.21.15.112:0/3344034970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280add40 0x7f69280b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692659c640 1 -- 172.21.15.112:0/3344034970 >> v1:172.21.15.112:6789/0 conn(0x7f692806caf0 legacy=0x7f69280b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692659c640 1 -- 172.21.15.112:0/3344034970 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69280aede0 con 0x7f692809ded0 2024-04-25T00:56:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692559a640 1 -- 172.21.15.112:0/3344034970 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2048786735 0 0) 0x7f692809e2b0 con 0x7f692806caf0 2024-04-25T00:56:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f6925d9b640 1 --2- 172.21.15.112:0/3344034970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280add40 0x7f69280b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692659c640 1 --2- 172.21.15.112:0/3344034970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69280b4980 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6918009870 tx=0x7f691801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=528e238e99494c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692559a640 1 -- 172.21.15.112:0/3344034970 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180b0070 con 0x7f692809ded0 2024-04-25T00:56:30.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692559a640 1 -- 172.21.15.112:0/3344034970 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69180ab600 con 0x7f692809ded0 2024-04-25T00:56:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692559a640 1 -- 172.21.15.112:0/3344034970 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180ab9d0 con 0x7f692809ded0 2024-04-25T00:56:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 -- 172.21.15.112:0/3344034970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 msgr2=0x7f69280b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3344034970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69280b4980 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6918009870 tx=0x7f691801c440 comp rx=0 tx=0).stop 2024-04-25T00:56:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 -- 172.21.15.112:0/3344034970 shutdown_connections 2024-04-25T00:56:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3344034970 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f69280add40 0x7f69280b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3344034970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69280b4980 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 -- 172.21.15.112:0/3344034970 >> 172.21.15.112:0/3344034970 conn(0x7f69280b8c80 msgr2=0x7f69280b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 -- 172.21.15.112:0/3344034970 shutdown_connections 2024-04-25T00:56:30.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.898+0000 7f692cbe5640 1 -- 172.21.15.112:0/3344034970 wait complete. 2024-04-25T00:56:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 Processor -- start 2024-04-25T00:56:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 -- start start 2024-04-25T00:56:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692806caf0 0x7f692813c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69281380e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.905 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69280add40 0x7f692809aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f69281504f0 con 0x7f692809ded0 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f69280b4310 con 0x7f69280add40 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6928150670 con 0x7f692806caf0 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69281380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69281380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58010/0 (socket says 172.21.15.112:58010) 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 -- 172.21.15.112:0/3589502407 learned_addr learned my addr 172.21.15.112:0/3589502407 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 -- 172.21.15.112:0/3589502407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692806caf0 msgr2=0x7f692813c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692806caf0 0x7f692813c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 -- 172.21.15.112:0/3589502407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69280add40 msgr2=0x7f692809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6926d9d640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69280add40 0x7f692809aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69280add40 0x7f692809aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 -- 172.21.15.112:0/3589502407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692809cbb0 con 0x7f692809ded0 2024-04-25T00:56:30.906 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6925d9b640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69281380e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f691c00c7f0 tx=0x7f691c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6916ffd640 1 -- 172.21.15.112:0/3589502407 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c015070 con 0x7f692809ded0 2024-04-25T00:56:30.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6916ffd640 1 -- 172.21.15.112:0/3589502407 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c007a30 con 0x7f692809ded0 2024-04-25T00:56:30.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692809ce30 con 0x7f692809ded0 2024-04-25T00:56:30.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6916ffd640 1 -- 172.21.15.112:0/3589502407 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c007e20 con 0x7f692809ded0 2024-04-25T00:56:30.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f692809bb10 con 0x7f692809ded0 2024-04-25T00:56:30.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692806d800 con 0x7f692809ded0 2024-04-25T00:56:30.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.902+0000 7f6916ffd640 1 -- 172.21.15.112:0/3589502407 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f691c018070 con 0x7f692809ded0 2024-04-25T00:56:30.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.906+0000 7f6916ffd640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69080678c0 0x7f6908069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:30.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.906+0000 7f6916ffd640 1 -- 172.21.15.112:0/3589502407 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f691c085b50 con 0x7f692809ded0 2024-04-25T00:56:30.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.906+0000 7f692659c640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69080678c0 0x7f6908069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:30.908 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.906+0000 7f692659c640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69080678c0 0x7f6908069d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69180ad000 tx=0x7f6918008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:30.911 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:30.910+0000 7f6916ffd640 1 -- 172.21.15.112:0/3589502407 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f692809cbb0 con 0x7f692809ded0 2024-04-25T00:56:31.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.062+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f692804f5f0 con 0x7f692809ded0 2024-04-25T00:56:31.064 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.062+0000 7f6916ffd640 1 -- 172.21.15.112:0/3589502407 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f691c056720 con 0x7f692809ded0 2024-04-25T00:56:31.065 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 32 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69080678c0 msgr2=0x7f6908069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69080678c0 0x7f6908069d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69180ad000 tx=0x7f6918008040 comp rx=0 tx=0).stop 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 msgr2=0x7f69281380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69281380e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f691c00c7f0 tx=0x7f691c00ccb0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 shutdown_connections 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f69080678c0 0x7f6908069d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f69280add40 0x7f692809aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f692809ded0 0x7f69281380e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 --2- 172.21.15.112:0/3589502407 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f692806caf0 0x7f692813c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 >> 172.21.15.112:0/3589502407 conn(0x7f69280b8c80 msgr2=0x7f69280b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:31.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 shutdown_connections 2024-04-25T00:56:31.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.066+0000 7f692cbe5640 1 -- 172.21.15.112:0/3589502407 wait complete. 2024-04-25T00:56:31.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:56:31.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.186+0000 7f95ddd7f640 1 Processor -- start 2024-04-25T00:56:31.191 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.186+0000 7f95ddd7f640 1 -- start start 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95ddd7f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f95d80affb0 0x7f95d809da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95ddd7f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f95d809dfa0 0x7f95d8148800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95ddd7f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95d80b1390 con 0x7f95d80b0390 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95ddd7f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f95d80b1d70 con 0x7f95d809dfa0 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95ddd7f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f95d80b1600 con 0x7f95d80affb0 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d7fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f95d80b0390 0x7f95d8145190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59020/0 (socket says 172.21.15.112:59020) 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d7fff640 1 -- 172.21.15.112:0/1889621210 learned_addr learned my addr 172.21.15.112:0/1889621210 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d6ffd640 1 --2- 172.21.15.112:0/1889621210 >> v2:172.21.15.112:3300/0 conn(0x7f95d809dfa0 0x7f95d8148800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d77fe640 1 --2- 172.21.15.112:0/1889621210 >> v2:172.21.15.187:3300/0 conn(0x7f95d80affb0 0x7f95d809da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d6ffd640 1 -- 172.21.15.112:0/1889621210 >> v2:172.21.15.187:3300/0 conn(0x7f95d80affb0 msgr2=0x7f95d809da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d6ffd640 1 --2- 172.21.15.112:0/1889621210 >> v2:172.21.15.187:3300/0 conn(0x7f95d80affb0 0x7f95d809da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d6ffd640 1 -- 172.21.15.112:0/1889621210 >> v1:172.21.15.112:6789/0 conn(0x7f95d80b0390 legacy=0x7f95d8145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d6ffd640 1 -- 172.21.15.112:0/1889621210 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d80b1fa0 con 0x7f95d809dfa0 2024-04-25T00:56:31.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d67fc640 1 -- 172.21.15.112:0/1889621210 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814120742 0 0) 0x7f95d80b1390 con 0x7f95d80b0390 2024-04-25T00:56:31.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d6ffd640 1 --2- 172.21.15.112:0/1889621210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d809dfa0 0x7f95d8148800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f95d80b6670 tx=0x7f95c801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66fc5e04e949141 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d67fc640 1 -- 172.21.15.112:0/1889621210 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80ab5f0 con 0x7f95d809dfa0 2024-04-25T00:56:31.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d67fc640 1 -- 172.21.15.112:0/1889621210 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95c80b0070 con 0x7f95d809dfa0 2024-04-25T00:56:31.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.190+0000 7f95d67fc640 1 -- 172.21.15.112:0/1889621210 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c80ab9b0 con 0x7f95d809dfa0 2024-04-25T00:56:31.195 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- 172.21.15.112:0/1889621210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d809dfa0 msgr2=0x7f95d8148800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/1889621210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d809dfa0 0x7f95d8148800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f95d80b6670 tx=0x7f95c801c3f0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- 172.21.15.112:0/1889621210 shutdown_connections 2024-04-25T00:56:31.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/1889621210 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d809dfa0 0x7f95d8148800 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/1889621210 >> v2:172.21.15.187:3300/0 conn(0x7f95d80affb0 0x7f95d809da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.197 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- 172.21.15.112:0/1889621210 >> 172.21.15.112:0/1889621210 conn(0x7f95d80b8c80 msgr2=0x7f95d80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- 172.21.15.112:0/1889621210 shutdown_connections 2024-04-25T00:56:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- 172.21.15.112:0/1889621210 wait complete. 2024-04-25T00:56:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 Processor -- start 2024-04-25T00:56:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- start start 2024-04-25T00:56:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95d809dfa0 0x7f95d80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d80affb0 0x7f95d81e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.200 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80b0390 0x7f95d814fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8152a90 con 0x7f95d80b0390 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8149150 con 0x7f95d80affb0 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f95d8152c10 con 0x7f95d809dfa0 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80b0390 0x7f95d814fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80b0390 0x7f95d814fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58022/0 (socket says 172.21.15.112:58022) 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 -- 172.21.15.112:0/4025541241 learned_addr learned my addr 172.21.15.112:0/4025541241 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 -- 172.21.15.112:0/4025541241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95d809dfa0 msgr2=0x7f95d80706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95d809dfa0 0x7f95d80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 -- 172.21.15.112:0/4025541241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d80affb0 msgr2=0x7f95d81e3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d6ffd640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d80affb0 0x7f95d81e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d80affb0 0x7f95d81e3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 -- 172.21.15.112:0/4025541241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d814e630 con 0x7f95d80b0390 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d6ffd640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d80affb0 0x7f95d81e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95d7fff640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80b0390 0x7f95d814fb60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c780 tx=0x7f95d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.201 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95cffff640 1 -- 172.21.15.112:0/4025541241 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0018070 con 0x7f95d80b0390 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95cffff640 1 -- 172.21.15.112:0/4025541241 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d000a640 con 0x7f95d80b0390 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d8070ed0 con 0x7f95d80b0390 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95cffff640 1 -- 172.21.15.112:0/4025541241 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d000aab0 con 0x7f95d80b0390 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d80711b0 con 0x7f95d80b0390 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.194+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d80b1040 con 0x7f95d80b0390 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.198+0000 7f95cffff640 1 -- 172.21.15.112:0/4025541241 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f95d001b070 con 0x7f95d80b0390 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.198+0000 7f95cffff640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f95b80678f0 0x7f95b8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.198+0000 7f95d77fe640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f95b80678f0 0x7f95b8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.198+0000 7f95cffff640 1 -- 172.21.15.112:0/4025541241 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f95d00851f0 con 0x7f95d80b0390 2024-04-25T00:56:31.202 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.198+0000 7f95d77fe640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f95b80678f0 0x7f95b8069db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f95d806f8d0 tx=0x7f95c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.203 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.202+0000 7f95cffff640 1 -- 172.21.15.112:0/4025541241 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f95d0055dc0 con 0x7f95d80b0390 2024-04-25T00:56:31.350 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.346+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f95d804f5f0 con 0x7f95d80b0390 2024-04-25T00:56:31.351 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95cffff640 1 -- 172.21.15.112:0/4025541241 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f95d00556c0 con 0x7f95d80b0390 2024-04-25T00:56:31.352 INFO:teuthology.orchestra.run.smithi112.stdout:pg_num: 64 2024-04-25T00:56:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f95b80678f0 msgr2=0x7f95b8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f95b80678f0 0x7f95b8069db0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f95d806f8d0 tx=0x7f95c0008040 comp rx=0 tx=0).stop 2024-04-25T00:56:31.353 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80b0390 msgr2=0x7f95d814fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80b0390 0x7f95d814fb60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f95d000c780 tx=0x7f95d000cc40 comp rx=0 tx=0).stop 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 shutdown_connections 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95d80b0390 0x7f95d814fb60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f95b80678f0 0x7f95b8069db0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f95d80affb0 0x7f95d81e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 --2- 172.21.15.112:0/4025541241 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f95d809dfa0 0x7f95d80706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 >> 172.21.15.112:0/4025541241 conn(0x7f95d80b8c80 msgr2=0x7f95d80b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 shutdown_connections 2024-04-25T00:56:31.354 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.350+0000 7f95ddd7f640 1 -- 172.21.15.112:0/4025541241 wait complete. 2024-04-25T00:56:31.368 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:31.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.478+0000 7f1f84b8e640 1 Processor -- start 2024-04-25T00:56:31.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.478+0000 7f1f84b8e640 1 -- start start 2024-04-25T00:56:31.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1f800b23f0 0x7f1f800add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f800b3ce0 0x7f1f8014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f800aeec0 con 0x7f1f80145200 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1f800af780 con 0x7f1f800b3ce0 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1f800b27d0 con 0x7f1f800b23f0 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f800b3ce0 0x7f1f8014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7ed9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f800b3ce0 0x7f1f8014fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58028/0 (socket says 172.21.15.112:58028) 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7ed9d640 1 -- 172.21.15.112:0/3912693135 learned_addr learned my addr 172.21.15.112:0/3912693135 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7e59c640 1 --2- 172.21.15.112:0/3912693135 >> v2:172.21.15.187:3300/0 conn(0x7f1f800b23f0 0x7f1f800add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7ed9d640 1 -- 172.21.15.112:0/3912693135 >> v2:172.21.15.187:3300/0 conn(0x7f1f800b23f0 msgr2=0x7f1f800add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7ed9d640 1 --2- 172.21.15.112:0/3912693135 >> v2:172.21.15.187:3300/0 conn(0x7f1f800b23f0 0x7f1f800add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7ed9d640 1 -- 172.21.15.112:0/3912693135 >> v1:172.21.15.112:6789/0 conn(0x7f1f80145200 legacy=0x7f1f80072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.484 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7ed9d640 1 -- 172.21.15.112:0/3912693135 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f800afa20 con 0x7f1f800b3ce0 2024-04-25T00:56:31.485 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7ed9d640 1 --2- 172.21.15.112:0/3912693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b3ce0 0x7f1f8014fb40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1f8006d9f0 tx=0x7f1f7801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcebb9ab3c2a43a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7d59a640 1 -- 172.21.15.112:0/3912693135 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f780a57d0 con 0x7f1f800b3ce0 2024-04-25T00:56:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7d59a640 1 -- 172.21.15.112:0/3912693135 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f780a5970 con 0x7f1f800b3ce0 2024-04-25T00:56:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f7d59a640 1 -- 172.21.15.112:0/3912693135 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f780a5d40 con 0x7f1f800b3ce0 2024-04-25T00:56:31.486 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3912693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b3ce0 msgr2=0x7f1f8014fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3912693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b3ce0 0x7f1f8014fb40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1f8006d9f0 tx=0x7f1f7801c9f0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3912693135 shutdown_connections 2024-04-25T00:56:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3912693135 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b3ce0 0x7f1f8014fb40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3912693135 >> v2:172.21.15.187:3300/0 conn(0x7f1f800b23f0 0x7f1f800add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3912693135 >> 172.21.15.112:0/3912693135 conn(0x7f1f800b5980 msgr2=0x7f1f800b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3912693135 shutdown_connections 2024-04-25T00:56:31.487 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.482+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3912693135 wait complete. 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 Processor -- start 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 -- start start 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b23f0 0x7f1f80139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f800b3ce0 0x7f1f801dda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f80145200 0x7f1f801ddf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800b52d0 con 0x7f1f800b23f0 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1f800b5150 con 0x7f1f800b3ce0 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1f801528a0 con 0x7f1f80145200 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b23f0 0x7f1f80139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b23f0 0x7f1f80139ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58040/0 (socket says 172.21.15.112:58040) 2024-04-25T00:56:31.489 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 -- 172.21.15.112:0/3506266110 learned_addr learned my addr 172.21.15.112:0/3506266110 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7dd9b640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f800b3ce0 0x7f1f801dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7ed9d640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f80145200 0x7f1f801ddf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 -- 172.21.15.112:0/3506266110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f80145200 msgr2=0x7f1f801ddf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f80145200 0x7f1f801ddf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 -- 172.21.15.112:0/3506266110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f800b3ce0 msgr2=0x7f1f801dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f800b3ce0 0x7f1f801dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 -- 172.21.15.112:0/3506266110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f80071fe0 con 0x7f1f800b23f0 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f7e59c640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b23f0 0x7f1f80139ad0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400d800 tx=0x7f1f7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f6effd640 1 -- 172.21.15.112:0/3506266110 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400b660 con 0x7f1f800b23f0 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f6effd640 1 -- 172.21.15.112:0/3506266110 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f74019070 con 0x7f1f800b23f0 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f801e1770 con 0x7f1f800b23f0 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f6effd640 1 -- 172.21.15.112:0/3506266110 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400ba00 con 0x7f1f800b23f0 2024-04-25T00:56:31.490 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f80071d00 con 0x7f1f800b23f0 2024-04-25T00:56:31.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8004f580 con 0x7f1f800b23f0 2024-04-25T00:56:31.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.486+0000 7f1f6effd640 1 -- 172.21.15.112:0/3506266110 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f7400bba0 con 0x7f1f800b23f0 2024-04-25T00:56:31.491 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.490+0000 7f1f6effd640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1f600678c0 0x7f1f60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.490+0000 7f1f7dd9b640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1f600678c0 0x7f1f60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.490+0000 7f1f6effd640 1 -- 172.21.15.112:0/3506266110 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f1f74086090 con 0x7f1f800b23f0 2024-04-25T00:56:31.492 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.490+0000 7f1f7dd9b640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1f600678c0 0x7f1f60069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f801deb40 tx=0x7f1f70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.494 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.494+0000 7f1f6effd640 1 -- 172.21.15.112:0/3506266110 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1f74056c60 con 0x7f1f800b23f0 2024-04-25T00:56:31.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.686+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f80142ee0 con 0x7f1f800b23f0 2024-04-25T00:56:31.689 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.686+0000 7f1f6effd640 1 -- 172.21.15.112:0/3506266110 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3537 (secure 0 0 0) 0x7f1f74056560 con 0x7f1f800b23f0 2024-04-25T00:56:31.689 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:31.690 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3898281312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3898281312},{"type":"v1","addr":"172.21.15.187:6833","nonce":3898281312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:56:18.588797+0000","modified":"2024-04-25T00:56:21.588160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3222796217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3222796217},{"type":"v1","addr":"172.21.15.112:6835","nonce":3222796217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-04-25T00:56:31.690 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1f600678c0 msgr2=0x7f1f60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1f600678c0 0x7f1f60069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f801deb40 tx=0x7f1f70009040 comp rx=0 tx=0).stop 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b23f0 msgr2=0x7f1f80139ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b23f0 0x7f1f80139ad0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400d800 tx=0x7f1f7400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 shutdown_connections 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1f80145200 0x7f1f801ddf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1f600678c0 0x7f1f60069d80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1f800b3ce0 0x7f1f801dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 --2- 172.21.15.112:0/3506266110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800b23f0 0x7f1f80139ad0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.692 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 >> 172.21.15.112:0/3506266110 conn(0x7f1f800b5980 msgr2=0x7f1f8006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 shutdown_connections 2024-04-25T00:56:31.693 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.690+0000 7f1f84b8e640 1 -- 172.21.15.112:0/3506266110 wait complete. 2024-04-25T00:56:31.707 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:31.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.810+0000 7f6e6ea3c640 1 Processor -- start 2024-04-25T00:56:31.814 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.810+0000 7f6e6ea3c640 1 -- start start 2024-04-25T00:56:31.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6ea3c640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6e680add70 0x7f6e68145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6ea3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680b3ce0 0x7f6e6814fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6ea3c640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e680aeec0 con 0x7f6e680b23f0 2024-04-25T00:56:31.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6ea3c640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6e680af780 con 0x7f6e680add70 2024-04-25T00:56:31.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6ea3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6e680b27d0 con 0x7f6e680b3ce0 2024-04-25T00:56:31.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6cfd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680b3ce0 0x7f6e6814fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.815 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e677fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f6e680add70 0x7f6e68145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6cfd9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680b3ce0 0x7f6e6814fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:54624/0 (socket says 172.21.15.112:54624) 2024-04-25T00:56:31.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6cfd9640 1 -- 172.21.15.112:0/2532535351 learned_addr learned my addr 172.21.15.112:0/2532535351 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:31.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6cfd9640 1 -- 172.21.15.112:0/2532535351 >> v2:172.21.15.187:3300/0 conn(0x7f6e680add70 msgr2=0x7f6e68145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:31.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6cfd9640 1 --2- 172.21.15.112:0/2532535351 >> v2:172.21.15.187:3300/0 conn(0x7f6e680add70 0x7f6e68145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.816 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6cfd9640 1 -- 172.21.15.112:0/2532535351 >> v1:172.21.15.187:6789/0 conn(0x7f6e680b23f0 legacy=0x7f6e68072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6cfd9640 1 -- 172.21.15.112:0/2532535351 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e680b4150 con 0x7f6e680b3ce0 2024-04-25T00:56:31.817 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e677fe640 1 --2- 172.21.15.112:0/2532535351 >> v2:172.21.15.187:3300/0 conn(0x7f6e680add70 0x7f6e68145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:56:31.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e6cfd9640 1 --2- 172.21.15.112:0/2532535351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680b3ce0 0x7f6e6814fb40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009870 tx=0x7f6e6001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22e5a9279c28dee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.818 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.814+0000 7f6e66ffd640 1 -- 172.21.15.112:0/2532535351 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e600a7070 con 0x7f6e680b3ce0 2024-04-25T00:56:31.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e66ffd640 1 -- 172.21.15.112:0/2532535351 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e600b0070 con 0x7f6e680b3ce0 2024-04-25T00:56:31.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e66ffd640 1 -- 172.21.15.112:0/2532535351 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e600ab890 con 0x7f6e680b3ce0 2024-04-25T00:56:31.819 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/2532535351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680b3ce0 msgr2=0x7f6e6814fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/2532535351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680b3ce0 0x7f6e6814fb40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009870 tx=0x7f6e6001c560 comp rx=0 tx=0).stop 2024-04-25T00:56:31.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/2532535351 shutdown_connections 2024-04-25T00:56:31.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/2532535351 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680b3ce0 0x7f6e6814fb40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/2532535351 >> v2:172.21.15.187:3300/0 conn(0x7f6e680add70 0x7f6e68145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/2532535351 >> 172.21.15.112:0/2532535351 conn(0x7f6e680b5980 msgr2=0x7f6e680b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:31.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/2532535351 shutdown_connections 2024-04-25T00:56:31.820 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/2532535351 wait complete. 2024-04-25T00:56:31.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 Processor -- start 2024-04-25T00:56:31.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- start start 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680add70 0x7f6e6806f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e680b23f0 0x7f6e681395f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e680b3ce0 0x7f6e6806ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6e680b52d0 con 0x7f6e680b3ce0 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6e680b5150 con 0x7f6e680b23f0 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6e681528a0 con 0x7f6e680add70 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e680b3ce0 0x7f6e6806ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e680b3ce0 0x7f6e6806ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58054/0 (socket says 172.21.15.112:58054) 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 -- 172.21.15.112:0/868281086 learned_addr learned my addr 172.21.15.112:0/868281086 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e677fe640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e680b23f0 0x7f6e681395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e67fff640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680add70 0x7f6e6806f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 -- 172.21.15.112:0/868281086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680add70 msgr2=0x7f6e6806f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680add70 0x7f6e6806f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 -- 172.21.15.112:0/868281086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e680b23f0 msgr2=0x7f6e681395f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e680b23f0 0x7f6e681395f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 -- 172.21.15.112:0/868281086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6813f250 con 0x7f6e680b3ce0 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6cfd9640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e680b3ce0 0x7f6e6806ffa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e6001ac10 tx=0x7f6e60019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e64ff9640 1 -- 172.21.15.112:0/868281086 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e600bf410 con 0x7f6e680b3ce0 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e64ff9640 1 -- 172.21.15.112:0/868281086 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e600bf5b0 con 0x7f6e680b3ce0 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6813e4f0 con 0x7f6e680b3ce0 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e64ff9640 1 -- 172.21.15.112:0/868281086 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e600bf960 con 0x7f6e680b3ce0 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.818+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6813ed30 con 0x7f6e680b3ce0 2024-04-25T00:56:31.824 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.822+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e680b1310 con 0x7f6e680b3ce0 2024-04-25T00:56:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.822+0000 7f6e64ff9640 1 -- 172.21.15.112:0/868281086 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6e600bd020 con 0x7f6e680b3ce0 2024-04-25T00:56:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.822+0000 7f6e64ff9640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6e480678c0 0x7f6e48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.822+0000 7f6e64ff9640 1 -- 172.21.15.112:0/868281086 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f6e60120220 con 0x7f6e680b3ce0 2024-04-25T00:56:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.822+0000 7f6e67fff640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6e480678c0 0x7f6e48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:31.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.822+0000 7f6e67fff640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6e480678c0 0x7f6e48069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000f70 tx=0x7f6e50009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:31.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.826+0000 7f6e64ff9640 1 -- 172.21.15.112:0/868281086 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6e680b1310 con 0x7f6e680b3ce0 2024-04-25T00:56:31.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.974+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e6804f580 con 0x7f6e680b3ce0 2024-04-25T00:56:31.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e64ff9640 1 -- 172.21.15.112:0/868281086 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16717 (secure 0 0 0) 0x7f6e60016150 con 0x7f6e680b3ce0 2024-04-25T00:56:31.979 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:31.979 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:20.226650+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:56:16.783920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:56:17.569481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6e480678c0 msgr2=0x7f6e48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6e480678c0 0x7f6e48069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e50000f70 tx=0x7f6e50009040 comp rx=0 tx=0).stop 2024-04-25T00:56:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e680b3ce0 msgr2=0x7f6e6806ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:31.981 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e680b3ce0 0x7f6e6806ffa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6e6001ac10 tx=0x7f6e60019300 comp rx=0 tx=0).stop 2024-04-25T00:56:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 shutdown_connections 2024-04-25T00:56:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e680b3ce0 0x7f6e6806ffa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f6e480678c0 0x7f6e48069d80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f6e680b23f0 0x7f6e681395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 --2- 172.21.15.112:0/868281086 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f6e680add70 0x7f6e6806f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 >> 172.21.15.112:0/868281086 conn(0x7f6e680b5980 msgr2=0x7f6e6806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 shutdown_connections 2024-04-25T00:56:31.982 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:31.978+0000 7f6e6ea3c640 1 -- 172.21.15.112:0/868281086 wait complete. 2024-04-25T00:56:31.996 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:56:31.997 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:32.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7bad3a6640 1 Processor -- start 2024-04-25T00:56:32.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7bad3a6640 1 -- start start 2024-04-25T00:56:32.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7bad3a6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ba8072400 0x7f7ba80b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7bad3a6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba80727e0 con 0x7f7ba80b7030 2024-04-25T00:56:32.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7bad3a6640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ba80b29a0 con 0x7f7ba8148f30 2024-04-25T00:56:32.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7bad3a6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f7ba813fa70 con 0x7f7ba8072400 2024-04-25T00:56:32.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7ba67fc640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7ba80b7030 0x7f7ba8148820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:59034/0 (socket says 172.21.15.112:59034) 2024-04-25T00:56:32.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7ba67fc640 1 -- 172.21.15.112:0/2363719979 learned_addr learned my addr 172.21.15.112:0/2363719979 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:32.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7ba6ffd640 1 --2- 172.21.15.112:0/2363719979 >> v2:172.21.15.112:3300/0 conn(0x7f7ba8072400 0x7f7ba80b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7ba5ffb640 1 -- 172.21.15.112:0/2363719979 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393176874 0 0) 0x7f7ba80727e0 con 0x7f7ba80b7030 2024-04-25T00:56:32.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7ba5ffb640 1 -- 172.21.15.112:0/2363719979 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b90003530 con 0x7f7ba80b7030 2024-04-25T00:56:32.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7ba6ffd640 1 -- 172.21.15.112:0/2363719979 >> v1:172.21.15.187:6789/0 conn(0x7f7ba8148f30 legacy=0x7f7ba814fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7ba6ffd640 1 -- 172.21.15.112:0/2363719979 >> v1:172.21.15.112:6789/0 conn(0x7f7ba80b7030 legacy=0x7f7ba8148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.102+0000 7f7ba6ffd640 1 -- 172.21.15.112:0/2363719979 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba806ef80 con 0x7f7ba8072400 2024-04-25T00:56:32.109 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7ba6ffd640 1 --2- 172.21.15.112:0/2363719979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8072400 0x7f7ba80b4330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7b980097c0 tx=0x7f7b9801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3482afba0db3b2ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7ba5ffb640 1 -- 172.21.15.112:0/2363719979 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b980b0070 con 0x7f7ba8072400 2024-04-25T00:56:32.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7ba5ffb640 1 -- 172.21.15.112:0/2363719979 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b980ab600 con 0x7f7ba8072400 2024-04-25T00:56:32.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7ba5ffb640 1 -- 172.21.15.112:0/2363719979 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b980ab9d0 con 0x7f7ba8072400 2024-04-25T00:56:32.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 -- 172.21.15.112:0/2363719979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8072400 msgr2=0x7f7ba80b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 --2- 172.21.15.112:0/2363719979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8072400 0x7f7ba80b4330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7b980097c0 tx=0x7f7b9801c410 comp rx=0 tx=0).stop 2024-04-25T00:56:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 -- 172.21.15.112:0/2363719979 shutdown_connections 2024-04-25T00:56:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 --2- 172.21.15.112:0/2363719979 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba8072400 0x7f7ba80b4330 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 -- 172.21.15.112:0/2363719979 >> 172.21.15.112:0/2363719979 conn(0x7f7ba80b77a0 msgr2=0x7f7ba80b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:32.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 -- 172.21.15.112:0/2363719979 shutdown_connections 2024-04-25T00:56:32.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 -- 172.21.15.112:0/2363719979 wait complete. 2024-04-25T00:56:32.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 Processor -- start 2024-04-25T00:56:32.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.106+0000 7f7bad3a6640 1 -- start start 2024-04-25T00:56:32.117 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba8072400 0x7f7ba8137f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba80b7030 0x7f7ba8136730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba81377f0 0x7f7ba81e1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba8153ca0 con 0x7f7ba81377f0 2024-04-25T00:56:32.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba806f5d0 con 0x7f7ba80b7030 2024-04-25T00:56:32.118 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba8153e20 con 0x7f7ba8072400 2024-04-25T00:56:32.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba81377f0 0x7f7ba81e1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba81377f0 0x7f7ba81e1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58072/0 (socket says 172.21.15.112:58072) 2024-04-25T00:56:32.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 -- 172.21.15.112:0/95168344 learned_addr learned my addr 172.21.15.112:0/95168344 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:32.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba67fc640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba80b7030 0x7f7ba8136730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.119 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba6ffd640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba8072400 0x7f7ba8137f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 -- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba8072400 msgr2=0x7f7ba8137f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba8072400 0x7f7ba8137f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 -- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba80b7030 msgr2=0x7f7ba8136730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba80b7030 0x7f7ba8136730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 -- 172.21.15.112:0/95168344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba813c6c0 con 0x7f7ba81377f0 2024-04-25T00:56:32.120 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba67fc640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba80b7030 0x7f7ba8136730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba77fe640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba81377f0 0x7f7ba81e1120 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c730 tx=0x7f7ba000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7b8f7fe640 1 -- 172.21.15.112:0/95168344 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0003b80 con 0x7f7ba81377f0 2024-04-25T00:56:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7b8f7fe640 1 -- 172.21.15.112:0/95168344 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba0003d20 con 0x7f7ba81377f0 2024-04-25T00:56:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba8135f30 con 0x7f7ba81377f0 2024-04-25T00:56:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7b8f7fe640 1 -- 172.21.15.112:0/95168344 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0005310 con 0x7f7ba81377f0 2024-04-25T00:56:32.121 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba80ba4f0 con 0x7f7ba81377f0 2024-04-25T00:56:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba804f580 con 0x7f7ba81377f0 2024-04-25T00:56:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7b8f7fe640 1 -- 172.21.15.112:0/95168344 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7ba00056c0 con 0x7f7ba81377f0 2024-04-25T00:56:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7b8f7fe640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7b940678c0 0x7f7b94069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7b8f7fe640 1 -- 172.21.15.112:0/95168344 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f7ba0085c40 con 0x7f7ba81377f0 2024-04-25T00:56:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba6ffd640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7b940678c0 0x7f7b94069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.122 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.110+0000 7f7ba6ffd640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7b940678c0 0x7f7b94069d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7b980ad000 tx=0x7f7b98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.123 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.114+0000 7f7b8f7fe640 1 -- 172.21.15.112:0/95168344 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7ba0056810 con 0x7f7ba81377f0 2024-04-25T00:56:32.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ba81434c0 con 0x7f7ba81377f0 2024-04-25T00:56:32.300 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7b8f7fe640 1 -- 172.21.15.112:0/95168344 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3537 (secure 0 0 0) 0x7f7ba0056110 con 0x7f7ba81377f0 2024-04-25T00:56:32.300 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:32.300 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3898281312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3898281312},{"type":"v1","addr":"172.21.15.187:6833","nonce":3898281312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:56:18.588797+0000","modified":"2024-04-25T00:56:21.588160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3222796217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3222796217},{"type":"v1","addr":"172.21.15.112:6835","nonce":3222796217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-04-25T00:56:32.301 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7b940678c0 msgr2=0x7f7b94069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7b940678c0 0x7f7b94069d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7b980ad000 tx=0x7f7b98008040 comp rx=0 tx=0).stop 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba81377f0 msgr2=0x7f7ba81e1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba81377f0 0x7f7ba81e1120 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c730 tx=0x7f7ba000cbf0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 shutdown_connections 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ba81377f0 0x7f7ba81e1120 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7b940678c0 0x7f7b94069d80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7ba80b7030 0x7f7ba8136730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.302 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 --2- 172.21.15.112:0/95168344 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7ba8072400 0x7f7ba8137f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 >> 172.21.15.112:0/95168344 conn(0x7f7ba80b77a0 msgr2=0x7f7ba81439c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:32.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 shutdown_connections 2024-04-25T00:56:32.303 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.298+0000 7f7bad3a6640 1 -- 172.21.15.112:0/95168344 wait complete. 2024-04-25T00:56:32.317 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:56:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceb1c9f640 1 Processor -- start 2024-04-25T00:56:32.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceb1c9f640 1 -- start start 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceb1c9f640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fceac0b23f0 0x7fceac0add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceb1c9f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 0x7fceac14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceb1c9f640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fceac0aeec0 con 0x7fceac145200 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceb1c9f640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fceac0af780 con 0x7fceac0b23f0 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceb1c9f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fceac0b27d0 con 0x7fceac0b3ce0 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceab7fe640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fceac0b23f0 0x7fceac0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fcea3fff640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fceac145200 0x7fceac072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:60542/0 (socket says 172.21.15.112:60542) 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fcea3fff640 1 -- 172.21.15.112:0/1159381285 learned_addr learned my addr 172.21.15.112:0/1159381285 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceabfff640 1 --2- 172.21.15.112:0/1159381285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 0x7fceac14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceab7fe640 1 --2- 172.21.15.112:0/1159381285 >> v2:172.21.15.187:3300/0 conn(0x7fceac0b23f0 0x7fceac0add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:49770/0 (socket says 172.21.15.112:49770) 2024-04-25T00:56:32.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceab7fe640 1 -- 172.21.15.112:0/1159381285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 msgr2=0x7fceac14fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:32.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceab7fe640 1 --2- 172.21.15.112:0/1159381285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 0x7fceac14fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceab7fe640 1 -- 172.21.15.112:0/1159381285 >> v1:172.21.15.187:6789/0 conn(0x7fceac145200 legacy=0x7fceac072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.425 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.422+0000 7fceab7fe640 1 -- 172.21.15.112:0/1159381285 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac0afa20 con 0x7fceac0b23f0 2024-04-25T00:56:32.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 --2- 172.21.15.112:0/1159381285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0b23f0 0x7fceac0add70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0097c0 tx=0x7fce9c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf5f730429755e4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceaaffd640 1 -- 172.21.15.112:0/1159381285 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0b1070 con 0x7fceac0b23f0 2024-04-25T00:56:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceaaffd640 1 -- 172.21.15.112:0/1159381285 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9c0ac410 con 0x7fceac0b23f0 2024-04-25T00:56:32.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceaaffd640 1 -- 172.21.15.112:0/1159381285 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0ac7e0 con 0x7fceac0b23f0 2024-04-25T00:56:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- 172.21.15.112:0/1159381285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0b23f0 msgr2=0x7fceac0add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/1159381285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0b23f0 0x7fceac0add70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0097c0 tx=0x7fce9c01c4a0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- 172.21.15.112:0/1159381285 shutdown_connections 2024-04-25T00:56:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/1159381285 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 0x7fceac14fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/1159381285 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac0b23f0 0x7fceac0add70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- 172.21.15.112:0/1159381285 >> 172.21.15.112:0/1159381285 conn(0x7fceac0b5980 msgr2=0x7fceac0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- 172.21.15.112:0/1159381285 shutdown_connections 2024-04-25T00:56:32.428 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- 172.21.15.112:0/1159381285 wait complete. 2024-04-25T00:56:32.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 Processor -- start 2024-04-25T00:56:32.429 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- start start 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0b23f0 0x7fceac06f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 0x7fceac1396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac145200 0x7fceac1e10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fceac0b52d0 con 0x7fceac0b23f0 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fceac0b5150 con 0x7fceac145200 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceb1c9f640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fceac1528a0 con 0x7fceac0b3ce0 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0b23f0 0x7fceac06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0b23f0 0x7fceac06f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58078/0 (socket says 172.21.15.112:58078) 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 -- 172.21.15.112:0/4054676768 learned_addr learned my addr 172.21.15.112:0/4054676768 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceabfff640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac145200 0x7fceac1e10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fcea3fff640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 0x7fceac1396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.431 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 -- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 msgr2=0x7fceac1396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 0x7fceac1396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 -- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac145200 msgr2=0x7fceac1e10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac145200 0x7fceac1e10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceab7fe640 1 -- 172.21.15.112:0/4054676768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceac13f700 con 0x7fceac0b23f0 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.426+0000 7fceabfff640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac145200 0x7fceac1e10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fceab7fe640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0b23f0 0x7fceac06f720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fce9c01a0d0 tx=0x7fce9c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fcea8ff9640 1 -- 172.21.15.112:0/4054676768 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0b1040 con 0x7fceac0b23f0 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fcea8ff9640 1 -- 172.21.15.112:0/4054676768 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce9c0bc070 con 0x7fceac0b23f0 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceac070060 con 0x7fceac0b23f0 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fcea8ff9640 1 -- 172.21.15.112:0/4054676768 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0b5410 con 0x7fceac0b23f0 2024-04-25T00:56:32.432 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceac13ead0 con 0x7fceac0b23f0 2024-04-25T00:56:32.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceac070670 con 0x7fceac0b23f0 2024-04-25T00:56:32.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fcea8ff9640 1 -- 172.21.15.112:0/4054676768 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fce9c0b55b0 con 0x7fceac0b23f0 2024-04-25T00:56:32.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fcea8ff9640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fce8c0678c0 0x7fce8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fcea8ff9640 1 -- 172.21.15.112:0/4054676768 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fce9c11f7f0 con 0x7fceac0b23f0 2024-04-25T00:56:32.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fcea3fff640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fce8c0678c0 0x7fce8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.430+0000 7fcea3fff640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fce8c0678c0 0x7fce8c069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fceac149890 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.434+0000 7fcea8ff9640 1 -- 172.21.15.112:0/4054676768 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fce9c016090 con 0x7fceac0b23f0 2024-04-25T00:56:32.618 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.614+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fceac04f580 con 0x7fceac0b23f0 2024-04-25T00:56:32.619 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fcea8ff9640 1 -- 172.21.15.112:0/4054676768 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3537 (secure 0 0 0) 0x7fce9c0f02b0 con 0x7fceac0b23f0 2024-04-25T00:56:32.619 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:32.619 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3898281312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3898281312},{"type":"v1","addr":"172.21.15.187:6833","nonce":3898281312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:56:18.588797+0000","modified":"2024-04-25T00:56:21.588160+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4926},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4926":{"gid":4926,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6835/3222796217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":3222796217},{"type":"v1","addr":"172.21.15.112:6835","nonce":3222796217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4926,"qdb_cluster":[4926]},"id":2}]} 2024-04-25T00:56:32.620 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 16 2024-04-25T00:56:32.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fce8c0678c0 msgr2=0x7fce8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fce8c0678c0 0x7fce8c069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fceac149890 tx=0x7fce94009040 comp rx=0 tx=0).stop 2024-04-25T00:56:32.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0b23f0 msgr2=0x7fceac06f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0b23f0 0x7fceac06f720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fce9c01a0d0 tx=0x7fce9c019240 comp rx=0 tx=0).stop 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 shutdown_connections 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fce8c0678c0 0x7fce8c069d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fceac145200 0x7fceac1e10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fceac0b3ce0 0x7fceac1396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 --2- 172.21.15.112:0/4054676768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fceac0b23f0 0x7fceac06f720 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 >> 172.21.15.112:0/4054676768 conn(0x7fceac0b5980 msgr2=0x7fceac06c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 shutdown_connections 2024-04-25T00:56:32.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.618+0000 7fceb1c9f640 1 -- 172.21.15.112:0/4054676768 wait complete. 2024-04-25T00:56:32.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:32.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.742+0000 7f5a18367640 1 Processor -- start 2024-04-25T00:56:32.746 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.742+0000 7f5a18367640 1 -- start start 2024-04-25T00:56:32.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a18367640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a1006c520 0x7f5a100727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a18367640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a100b47d0 0x7f5a100b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a18367640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a100b5300 con 0x7f5a10141110 2024-04-25T00:56:32.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a18367640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f5a1006c900 con 0x7f5a100b47d0 2024-04-25T00:56:32.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a18367640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a100ae670 con 0x7f5a1006c520 2024-04-25T00:56:32.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a16103640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a1006c520 0x7f5a100727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a15902640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f5a100b47d0 0x7f5a100b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a16103640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a1006c520 0x7f5a100727d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3301/0 says I am v2:172.21.15.112:54668/0 (socket says 172.21.15.112:54668) 2024-04-25T00:56:32.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a16103640 1 -- 172.21.15.112:0/4122573075 learned_addr learned my addr 172.21.15.112:0/4122573075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:32.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a16103640 1 -- 172.21.15.112:0/4122573075 >> v2:172.21.15.187:3300/0 conn(0x7f5a100b47d0 msgr2=0x7f5a100b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:32.748 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a16103640 1 --2- 172.21.15.112:0/4122573075 >> v2:172.21.15.187:3300/0 conn(0x7f5a100b47d0 0x7f5a100b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a16103640 1 -- 172.21.15.112:0/4122573075 >> v1:172.21.15.187:6789/0 conn(0x7f5a10141110 legacy=0x7f5a101415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.749 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a16103640 1 -- 172.21.15.112:0/4122573075 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a100ae950 con 0x7f5a1006c520 2024-04-25T00:56:32.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a16103640 1 --2- 172.21.15.112:0/4122573075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a1006c520 0x7f5a100727d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009870 tx=0x7f5a0801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20233ee8d361ccf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a15101640 1 -- 172.21.15.112:0/4122573075 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080b0070 con 0x7f5a1006c520 2024-04-25T00:56:32.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a15101640 1 -- 172.21.15.112:0/4122573075 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a080ab600 con 0x7f5a1006c520 2024-04-25T00:56:32.750 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.746+0000 7f5a15101640 1 -- 172.21.15.112:0/4122573075 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080ab9d0 con 0x7f5a1006c520 2024-04-25T00:56:32.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- 172.21.15.112:0/4122573075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a1006c520 msgr2=0x7f5a100727d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 --2- 172.21.15.112:0/4122573075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a1006c520 0x7f5a100727d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009870 tx=0x7f5a0801c4d0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- 172.21.15.112:0/4122573075 shutdown_connections 2024-04-25T00:56:32.751 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 --2- 172.21.15.112:0/4122573075 >> v2:172.21.15.187:3300/0 conn(0x7f5a100b47d0 0x7f5a100b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 --2- 172.21.15.112:0/4122573075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a1006c520 0x7f5a100727d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- 172.21.15.112:0/4122573075 >> 172.21.15.112:0/4122573075 conn(0x7f5a100b5980 msgr2=0x7f5a100b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:32.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- 172.21.15.112:0/4122573075 shutdown_connections 2024-04-25T00:56:32.752 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- 172.21.15.112:0/4122573075 wait complete. 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 Processor -- start 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- start start 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1006c520 0x7f5a10130d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a100b47d0 0x7f5a10135780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a10141110 0x7f5a101e3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5a101548c0 con 0x7f5a1006c520 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f5a10154740 con 0x7f5a100b47d0 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5a10154a40 con 0x7f5a10141110 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1006c520 0x7f5a10130d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1006c520 0x7f5a10130d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:58086/0 (socket says 172.21.15.112:58086) 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 -- 172.21.15.112:0/1445827104 learned_addr learned my addr 172.21.15.112:0/1445827104 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:32.754 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a15902640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a100b47d0 0x7f5a10135780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16904640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a10141110 0x7f5a101e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 -- 172.21.15.112:0/1445827104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a10141110 msgr2=0x7f5a101e3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a10141110 0x7f5a101e3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 -- 172.21.15.112:0/1445827104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a100b47d0 msgr2=0x7f5a10135780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a100b47d0 0x7f5a10135780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 -- 172.21.15.112:0/1445827104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a10135ee0 con 0x7f5a1006c520 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a16103640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1006c520 0x7f5a10130d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5a080ae040 tx=0x7f5a0801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a06ffd640 1 -- 172.21.15.112:0/1445827104 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080ba020 con 0x7f5a1006c520 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a06ffd640 1 -- 172.21.15.112:0/1445827104 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a080b6420 con 0x7f5a1006c520 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a101361c0 con 0x7f5a1006c520 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a06ffd640 1 -- 172.21.15.112:0/1445827104 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a080b0040 con 0x7f5a1006c520 2024-04-25T00:56:32.755 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a10134a50 con 0x7f5a1006c520 2024-04-25T00:56:32.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.750+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1004f580 con 0x7f5a1006c520 2024-04-25T00:56:32.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.754+0000 7f5a06ffd640 1 -- 172.21.15.112:0/1445827104 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a080bd020 con 0x7f5a1006c520 2024-04-25T00:56:32.756 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.754+0000 7f5a06ffd640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59f40678c0 0x7f59f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:32.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.754+0000 7f5a06ffd640 1 -- 172.21.15.112:0/1445827104 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f5a0811fe20 con 0x7f5a1006c520 2024-04-25T00:56:32.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.754+0000 7f5a15902640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59f40678c0 0x7f59f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:32.757 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.754+0000 7f5a15902640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59f40678c0 0x7f59f4069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a100b2110 tx=0x7f5a00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:32.760 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.758+0000 7f5a06ffd640 1 -- 172.21.15.112:0/1445827104 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5a080c4030 con 0x7f5a1006c520 2024-04-25T00:56:32.907 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.906+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a10135cc0 con 0x7f5a1006c520 2024-04-25T00:56:32.910 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.906+0000 7f5a06ffd640 1 -- 172.21.15.112:0/1445827104 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16717 (secure 0 0 0) 0x7f5a08016150 con 0x7f5a1006c520 2024-04-25T00:56:32.910 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:32.911 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":29,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:20.226650+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:56:16.783920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T00:56:17.569481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:32.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59f40678c0 msgr2=0x7f59f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59f40678c0 0x7f59f4069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a100b2110 tx=0x7f5a00008040 comp rx=0 tx=0).stop 2024-04-25T00:56:32.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1006c520 msgr2=0x7f5a10130d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:32.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1006c520 0x7f5a10130d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5a080ae040 tx=0x7f5a0801ad10 comp rx=0 tx=0).stop 2024-04-25T00:56:32.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 shutdown_connections 2024-04-25T00:56:32.913 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5a10141110 0x7f5a101e3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f59f40678c0 0x7f59f4069d80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f5a100b47d0 0x7f5a10135780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 --2- 172.21.15.112:0/1445827104 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a1006c520 0x7f5a10130d00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:32.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 >> 172.21.15.112:0/1445827104 conn(0x7f5a100b5980 msgr2=0x7f5a100b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:32.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 shutdown_connections 2024-04-25T00:56:32.914 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:32.910+0000 7f5a18367640 1 -- 172.21.15.112:0/1445827104 wait complete. 2024-04-25T00:56:32.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:56:33.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5317de8640 1 Processor -- start 2024-04-25T00:56:33.049 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5317de8640 1 -- start start 2024-04-25T00:56:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5317de8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100adfd0 0x7f53100ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.050 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5317de8640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f53100ae9a0 0x7f53100b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5317de8640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53100b52f0 con 0x7f53100aee00 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5317de8640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f53100b0800 con 0x7f53100ae9a0 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5317de8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5310143f50 con 0x7f53100adfd0 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5315383640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f53100ae9a0 0x7f53100b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5315383640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f53100ae9a0 0x7f53100b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60766/0 (socket says 172.21.15.112:60766) 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5315383640 1 -- 172.21.15.112:0/2948290206 learned_addr learned my addr 172.21.15.112:0/2948290206 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5315b84640 1 --2- 172.21.15.112:0/2948290206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100adfd0 0x7f53100ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5315383640 1 -- 172.21.15.112:0/2948290206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100adfd0 msgr2=0x7f53100ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5315383640 1 --2- 172.21.15.112:0/2948290206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100adfd0 0x7f53100ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5315383640 1 -- 172.21.15.112:0/2948290206 >> v1:172.21.15.187:6789/0 conn(0x7f53100aee00 legacy=0x7f53100b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.051 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.046+0000 7f5315383640 1 -- 172.21.15.112:0/2948290206 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5310144230 con 0x7f53100ae9a0 2024-04-25T00:56:33.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5315383640 1 --2- 172.21.15.112:0/2948290206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f53100b2240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f530000bb40 tx=0x7f530001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f543000a2e7cbac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:33.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5314b82640 1 -- 172.21.15.112:0/2948290206 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53000b4070 con 0x7f53100ae9a0 2024-04-25T00:56:33.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5314b82640 1 -- 172.21.15.112:0/2948290206 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53000a7b70 con 0x7f53100ae9a0 2024-04-25T00:56:33.053 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5314b82640 1 -- 172.21.15.112:0/2948290206 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53000af440 con 0x7f53100ae9a0 2024-04-25T00:56:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5317de8640 1 -- 172.21.15.112:0/2948290206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 msgr2=0x7f53100b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5317de8640 1 --2- 172.21.15.112:0/2948290206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f53100b2240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f530000bb40 tx=0x7f530001eb80 comp rx=0 tx=0).stop 2024-04-25T00:56:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5317de8640 1 -- 172.21.15.112:0/2948290206 shutdown_connections 2024-04-25T00:56:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5317de8640 1 --2- 172.21.15.112:0/2948290206 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f53100b2240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5317de8640 1 --2- 172.21.15.112:0/2948290206 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100adfd0 0x7f53100ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.054 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5317de8640 1 -- 172.21.15.112:0/2948290206 >> 172.21.15.112:0/2948290206 conn(0x7f53100b5980 msgr2=0x7f53100b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5317de8640 1 -- 172.21.15.112:0/2948290206 shutdown_connections 2024-04-25T00:56:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.050+0000 7f5317de8640 1 -- 172.21.15.112:0/2948290206 wait complete. 2024-04-25T00:56:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 Processor -- start 2024-04-25T00:56:33.055 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 -- start start 2024-04-25T00:56:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53100adfd0 0x7f53101dea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f5310070220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100aee00 0x7f5310133c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f5310150360 con 0x7f53100ae9a0 2024-04-25T00:56:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f53101501e0 con 0x7f53100adfd0 2024-04-25T00:56:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f53101504e0 con 0x7f53100aee00 2024-04-25T00:56:33.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f5310070220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f5310070220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60770/0 (socket says 172.21.15.112:60770) 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 -- 172.21.15.112:0/3163324297 learned_addr learned my addr 172.21.15.112:0/3163324297 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315b84640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53100adfd0 0x7f53101dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5316385640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100aee00 0x7f5310133c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 -- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100aee00 msgr2=0x7f5310133c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100aee00 0x7f5310133c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 -- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53100adfd0 msgr2=0x7f53101dea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53100adfd0 0x7f53101dea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 -- 172.21.15.112:0/3163324297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5310140300 con 0x7f53100ae9a0 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315b84640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53100adfd0 0x7f53101dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5315383640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f5310070220 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f530000b2c0 tx=0x7f530001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f53067fc640 1 -- 172.21.15.112:0/3163324297 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53000b4040 con 0x7f53100ae9a0 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f53067fc640 1 -- 172.21.15.112:0/3163324297 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53000a7970 con 0x7f53100ae9a0 2024-04-25T00:56:33.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5310071c90 con 0x7f53100ae9a0 2024-04-25T00:56:33.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f53067fc640 1 -- 172.21.15.112:0/3163324297 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53000ba680 con 0x7f53100ae9a0 2024-04-25T00:56:33.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53101def90 con 0x7f53100ae9a0 2024-04-25T00:56:33.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53101df1c0 con 0x7f53100ae9a0 2024-04-25T00:56:33.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.054+0000 7f53067fc640 1 -- 172.21.15.112:0/3163324297 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f53000c1020 con 0x7f53100ae9a0 2024-04-25T00:56:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.058+0000 7f53067fc640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52f40678f0 0x7f52f4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.058+0000 7f5315b84640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52f40678f0 0x7f52f4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.058+0000 7f53067fc640 1 -- 172.21.15.112:0/3163324297 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f5300122ab0 con 0x7f53100ae9a0 2024-04-25T00:56:33.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.058+0000 7f5315b84640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52f40678f0 0x7f52f4069db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f530c009840 tx=0x7f530c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:33.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.062+0000 7f53067fc640 1 -- 172.21.15.112:0/3163324297 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f53101df1c0 con 0x7f53100ae9a0 2024-04-25T00:56:33.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.258+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f531004f580 con 0x7f53100ae9a0 2024-04-25T00:56:33.260 INFO:tasks.ceph.mon.a.smithi112.stderr:2024-04-25T00:56:33.258+0000 7f8589c5c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:56:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.258+0000 7f53067fc640 1 -- 172.21.15.112:0/3163324297 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f5310071c90 con 0x7f53100ae9a0 2024-04-25T00:56:33.262 INFO:teuthology.orchestra.run.smithi112.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:56:33.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52f40678f0 msgr2=0x7f52f4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52f40678f0 0x7f52f4069db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f530c009840 tx=0x7f530c008040 comp rx=0 tx=0).stop 2024-04-25T00:56:33.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 msgr2=0x7f5310070220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f5310070220 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f530000b2c0 tx=0x7f530001b920 comp rx=0 tx=0).stop 2024-04-25T00:56:33.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 shutdown_connections 2024-04-25T00:56:33.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f53100aee00 0x7f5310133c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f53100ae9a0 0x7f5310070220 secure :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f530000b2c0 tx=0x7f530001b920 comp rx=0 tx=0).stop 2024-04-25T00:56:33.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52f40678f0 0x7f52f4069db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 --2- 172.21.15.112:0/3163324297 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f53100adfd0 0x7f53101dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.262+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 >> 172.21.15.112:0/3163324297 conn(0x7f53100b5980 msgr2=0x7f53100b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:33.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.266+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 shutdown_connections 2024-04-25T00:56:33.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.266+0000 7f5317de8640 1 -- 172.21.15.112:0/3163324297 wait complete. 2024-04-25T00:56:33.267 INFO:tasks.ceph.mds.a.smithi112.stderr: -2411> 2024-04-25T00:56:23.023+0000 7faff1480640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:56:33.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:56:33.300 INFO:tasks.ceph.mds.a.smithi112.stdout:starting mds.a at 2024-04-25T00:56:33.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2eec39640 1 Processor -- start 2024-04-25T00:56:33.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2eec39640 1 -- start start 2024-04-25T00:56:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2eec39640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e80724a0 0x7fe2e806ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2eec39640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2e8140b00 0x7fe2e8140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2eec39640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2e806d940 con 0x7fe2e806d560 2024-04-25T00:56:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2eec39640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fe2e8072010 con 0x7fe2e8140b00 2024-04-25T00:56:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2eec39640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e8071cc0 con 0x7fe2e80724a0 2024-04-25T00:56:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2ed1d6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2e8140b00 0x7fe2e8140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2e7fff640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e80724a0 0x7fe2e806ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.399 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2ed1d6640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fe2e8140b00 0x7fe2e8140fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40002/0 (socket says 172.21.15.112:40002) 2024-04-25T00:56:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2ec9d5640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7fe2e806d560 0x7fe2e814aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:38788/0 (socket says 172.21.15.112:38788) 2024-04-25T00:56:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2ed1d6640 1 -- 172.21.15.112:0/2484615078 learned_addr learned my addr 172.21.15.112:0/2484615078 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2e7fff640 1 -- 172.21.15.112:0/2484615078 >> v2:172.21.15.187:3300/0 conn(0x7fe2e8140b00 msgr2=0x7fe2e8140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2e7fff640 1 --2- 172.21.15.112:0/2484615078 >> v2:172.21.15.187:3300/0 conn(0x7fe2e8140b00 0x7fe2e8140fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2e7fff640 1 -- 172.21.15.112:0/2484615078 >> v1:172.21.15.187:6789/0 conn(0x7fe2e806d560 legacy=0x7fe2e814aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2e7fff640 1 -- 172.21.15.112:0/2484615078 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e8071550 con 0x7fe2e80724a0 2024-04-25T00:56:33.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.394+0000 7fe2ed1d6640 1 --2- 172.21.15.112:0/2484615078 >> v2:172.21.15.187:3300/0 conn(0x7fe2e8140b00 0x7fe2e8140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:56:33.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2e7fff640 1 --2- 172.21.15.112:0/2484615078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e80724a0 0x7fe2e806ff40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0097c0 tx=0x7fe2dc01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65ad91d73604764 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:33.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2e77fe640 1 -- 172.21.15.112:0/2484615078 <== mon.2 v2:172.21.15.187:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0ab600 con 0x7fe2e80724a0 2024-04-25T00:56:33.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2e77fe640 1 -- 172.21.15.112:0/2484615078 <== mon.2 v2:172.21.15.187:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0b0070 con 0x7fe2e80724a0 2024-04-25T00:56:33.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2e77fe640 1 -- 172.21.15.112:0/2484615078 <== mon.2 v2:172.21.15.187:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0ab9c0 con 0x7fe2e80724a0 2024-04-25T00:56:33.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- 172.21.15.112:0/2484615078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e80724a0 msgr2=0x7fe2e806ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 --2- 172.21.15.112:0/2484615078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e80724a0 0x7fe2e806ff40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0097c0 tx=0x7fe2dc01c3e0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- 172.21.15.112:0/2484615078 shutdown_connections 2024-04-25T00:56:33.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 --2- 172.21.15.112:0/2484615078 >> v2:172.21.15.187:3300/0 conn(0x7fe2e8140b00 0x7fe2e8140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 --2- 172.21.15.112:0/2484615078 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e80724a0 0x7fe2e806ff40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- 172.21.15.112:0/2484615078 >> 172.21.15.112:0/2484615078 conn(0x7fe2e80b6840 msgr2=0x7fe2e80b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:33.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- 172.21.15.112:0/2484615078 shutdown_connections 2024-04-25T00:56:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- 172.21.15.112:0/2484615078 wait complete. 2024-04-25T00:56:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 Processor -- start 2024-04-25T00:56:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- start start 2024-04-25T00:56:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e806d560 0x7fe2e80ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e80724a0 0x7fe2e813fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2e8140b00 0x7fe2e80af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8154890 con 0x7fe2e80724a0 2024-04-25T00:56:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8071190 con 0x7fe2e8140b00 2024-04-25T00:56:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.398+0000 7fe2eec39640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e8154a10 con 0x7fe2e806d560 2024-04-25T00:56:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e80724a0 0x7fe2e813fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e80724a0 0x7fe2e813fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60804/0 (socket says 172.21.15.112:60804) 2024-04-25T00:56:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 -- 172.21.15.112:0/496431809 learned_addr learned my addr 172.21.15.112:0/496431809 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2ec9d5640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e806d560 0x7fe2e80ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2ed1d6640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2e8140b00 0x7fe2e80af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 -- 172.21.15.112:0/496431809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e806d560 msgr2=0x7fe2e80ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e806d560 0x7fe2e80ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 -- 172.21.15.112:0/496431809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2e8140b00 msgr2=0x7fe2e80af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2e8140b00 0x7fe2e80af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:33.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 -- 172.21.15.112:0/496431809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e80b03b0 con 0x7fe2e80724a0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e7fff640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e80724a0 0x7fe2e813fe90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc01a010 tx=0x7fe2dc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e57fa640 1 -- 172.21.15.112:0/496431809 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0bf610 con 0x7fe2e80724a0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e57fa640 1 -- 172.21.15.112:0/496431809 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0bf7b0 con 0x7fe2e80724a0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e80b0690 con 0x7fe2e80724a0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e57fa640 1 -- 172.21.15.112:0/496431809 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0bfba0 con 0x7fe2e80724a0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e80afdc0 con 0x7fe2e80724a0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e80b3de0 con 0x7fe2e80724a0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e57fa640 1 -- 172.21.15.112:0/496431809 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe2dc0bd020 con 0x7fe2e80724a0 2024-04-25T00:56:33.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e57fa640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe2d40678c0 0x7fe2d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:33.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2ec9d5640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe2d40678c0 0x7fe2d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:33.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2e57fa640 1 -- 172.21.15.112:0/496431809 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7fe2dc120330 con 0x7fe2e80724a0 2024-04-25T00:56:33.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.402+0000 7fe2ec9d5640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe2d40678c0 0x7fe2d4069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8000f70 tx=0x7fe2d8007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:33.414 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.406+0000 7fe2e57fa640 1 -- 172.21.15.112:0/496431809 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe2dc016090 con 0x7fe2e80724a0 2024-04-25T00:56:33.590 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:33.586+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe2e804f5f0 con 0x7fe2e80724a0 2024-04-25T00:56:34.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.262+0000 7fe2e57fa640 1 -- 172.21.15.112:0/496431809 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7fe2dc0f0df0 con 0x7fe2e80724a0 2024-04-25T00:56:34.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe2d40678c0 msgr2=0x7fe2d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:34.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe2d40678c0 0x7fe2d4069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8000f70 tx=0x7fe2d8007c90 comp rx=0 tx=0).stop 2024-04-25T00:56:34.269 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e80724a0 msgr2=0x7fe2e813fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:34.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e80724a0 0x7fe2e813fe90 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc01a010 tx=0x7fe2dc01ac20 comp rx=0 tx=0).stop 2024-04-25T00:56:34.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 shutdown_connections 2024-04-25T00:56:34.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fe2d40678c0 0x7fe2d4069d80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fe2e8140b00 0x7fe2e80af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe2e80724a0 0x7fe2e813fe90 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.270 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 --2- 172.21.15.112:0/496431809 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fe2e806d560 0x7fe2e80ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 >> 172.21.15.112:0/496431809 conn(0x7fe2e80b6840 msgr2=0x7fe2e80676a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:34.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 shutdown_connections 2024-04-25T00:56:34.271 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.266+0000 7fe2eec39640 1 -- 172.21.15.112:0/496431809 wait complete. 2024-04-25T00:56:34.292 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:56:34.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f40cb7640 1 Processor -- start 2024-04-25T00:56:34.464 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f40cb7640 1 -- start start 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f40cb7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c13f670 0x7f9f3c150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f40cb7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3c0b25c0 con 0x7f9f3c0b4d90 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f40cb7640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f3c06f630 con 0x7f9f3c0b21e0 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f40cb7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c06f100 con 0x7f9f3c13f670 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f33fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9f3c0b4d90 0x7f9f3c13ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32808/0 (socket says 172.21.15.112:32808) 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f33fff640 1 -- 172.21.15.112:0/909829994 learned_addr learned my addr 172.21.15.112:0/909829994 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f3ad9d640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c13f670 0x7f9f3c150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152462702 0 0) 0x7f9f3c0b25c0 con 0x7f9f3c0b4d90 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f28003530 con 0x7f9f3c0b4d90 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 886355530 0 0) 0x7f9f28003530 con 0x7f9f3c0b4d90 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c13f670 msgr2=0x7f9f3c150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:34.467 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c13f670 0x7f9f3c150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 >> v1:172.21.15.187:6789/0 conn(0x7f9f3c0b21e0 legacy=0x7f9f3c148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c06f350 con 0x7f9f3c0b4d90 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f9f24001680 con 0x7f9f3c0b4d90 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 >> v1:172.21.15.112:6789/0 conn(0x7f9f3c0b4d90 legacy=0x7f9f3c13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f280039f0 0x7f9f28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f28004810 0x7f9f28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c13f670 0x7f9f28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c06f100 con 0x7f9f28004810 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2800bb00 con 0x7f9f280039f0 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2800bd00 con 0x7f9f3c13f670 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f33fff640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f28004810 0x7f9f28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:34.468 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f3a59c640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f280039f0 0x7f9f28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f3ad9d640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c13f670 0x7f9f28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f33fff640 1 -- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c13f670 msgr2=0x7f9f28005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f33fff640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c13f670 0x7f9f28005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f33fff640 1 -- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f280039f0 msgr2=0x7f9f28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f33fff640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f280039f0 0x7f9f28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.469 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.462+0000 7f9f33fff640 1 -- 172.21.15.112:0/909829994 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f28003530 con 0x7f9f28004810 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f33fff640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f28004810 0x7f9f28004c90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c1462a0 tx=0x7f9f2401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b148e3290b5267e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24007cf0 con 0x7f9f28004810 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f240a9070 con 0x7f9f28004810 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f39d9b640 1 -- 172.21.15.112:0/909829994 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24005d20 con 0x7f9f28004810 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f40cb7640 1 -- 172.21.15.112:0/909829994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f28004810 msgr2=0x7f9f28004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f40cb7640 1 --2- 172.21.15.112:0/909829994 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f28004810 0x7f9f28004c90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c1462a0 tx=0x7f9f2401cba0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f3ad9d640 1 -- 172.21.15.112:0/909829994 reap_dead start 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f40cb7640 1 -- 172.21.15.112:0/909829994 shutdown_connections 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f40cb7640 1 -- 172.21.15.112:0/909829994 >> 172.21.15.112:0/909829994 conn(0x7f9f3c0b8930 msgr2=0x7f9f3c0b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f40cb7640 1 -- 172.21.15.112:0/909829994 shutdown_connections 2024-04-25T00:56:34.473 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f40cb7640 1 -- 172.21.15.112:0/909829994 wait complete. 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f40cb7640 1 Processor -- start 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.466+0000 7f9f40cb7640 1 -- start start 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f28004810 0x7f9f3c142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f3c1ddc40 0x7f9f3c071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c072510 0x7f9f3c145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0b4b10 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c0b4990 con 0x7f9f28004810 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3c1512b0 con 0x7f9f3c072510 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f3c1ddc40 0x7f9f3c071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f3c1ddc40 0x7f9f3c071fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60818/0 (socket says 172.21.15.112:60818) 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 -- 172.21.15.112:0/2631619279 learned_addr learned my addr 172.21.15.112:0/2631619279 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:34.474 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f3a59c640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f28004810 0x7f9f3c142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f3ad9d640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c072510 0x7f9f3c145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 -- 172.21.15.112:0/2631619279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c072510 msgr2=0x7f9f3c145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c072510 0x7f9f3c145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 -- 172.21.15.112:0/2631619279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f28004810 msgr2=0x7f9f3c142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f28004810 0x7f9f3c142fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 -- 172.21.15.112:0/2631619279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c1446d0 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f33fff640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f3c1ddc40 0x7f9f3c071fd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f2401c9d0 tx=0x7f9f28005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f32ffd640 1 -- 172.21.15.112:0/2631619279 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f240047f0 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f32ffd640 1 -- 172.21.15.112:0/2631619279 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f24004990 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c131ba0 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f32ffd640 1 -- 172.21.15.112:0/2631619279 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24004d40 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.475 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c13a2b0 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3c068260 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.476 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f32ffd640 1 -- 172.21.15.112:0/2631619279 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9f240a5790 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f32ffd640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9f1c0678c0 0x7f9f1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:34.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f32ffd640 1 -- 172.21.15.112:0/2631619279 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5795+0+0 (secure 0 0 0) 0x7f9f2411f470 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f3a59c640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9f1c0678c0 0x7f9f1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:34.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.470+0000 7f9f3a59c640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9f1c0678c0 0x7f9f1c069d80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005400 tx=0x7f9f2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:34.478 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.474+0000 7f9f32ffd640 1 -- 172.21.15.112:0/2631619279 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9f240168b0 con 0x7f9f3c1ddc40 2024-04-25T00:56:34.640 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:34.638+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9f3c13e680 con 0x7f9f3c1ddc40 2024-04-25T00:56:35.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.610+0000 7f9f32ffd640 1 -- 172.21.15.112:0/2631619279 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f9f240eff50 con 0x7f9f3c1ddc40 2024-04-25T00:56:35.615 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:56:35.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9f1c0678c0 msgr2=0x7f9f1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:35.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9f1c0678c0 0x7f9f1c069d80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c005400 tx=0x7f9f2c009040 comp rx=0 tx=0).stop 2024-04-25T00:56:35.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f3c1ddc40 msgr2=0x7f9f3c071fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f3c1ddc40 0x7f9f3c071fd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f2401c9d0 tx=0x7f9f28005840 comp rx=0 tx=0).stop 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 shutdown_connections 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9f3c072510 0x7f9f3c145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9f3c1ddc40 0x7f9f3c071fd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9f1c0678c0 0x7f9f1c069d80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 --2- 172.21.15.112:0/2631619279 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9f28004810 0x7f9f3c142fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 >> 172.21.15.112:0/2631619279 conn(0x7f9f3c0b8930 msgr2=0x7f9f3c1359f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 shutdown_connections 2024-04-25T00:56:35.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.618+0000 7f9f40cb7640 1 -- 172.21.15.112:0/2631619279 wait complete. 2024-04-25T00:56:35.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:56:35.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632f1c5640 1 Processor -- start 2024-04-25T00:56:35.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632f1c5640 1 -- start start 2024-04-25T00:56:35.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632f1c5640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f632809f3e0 0x7f632809aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:35.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632f1c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 0x7f63280bbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:35.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632f1c5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f632809c080 0x7f63281412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:35.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632f1c5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f632809f7c0 con 0x7f632809c080 2024-04-25T00:56:35.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632f1c5640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f63280bacf0 con 0x7f632809f3e0 2024-04-25T00:56:35.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632f1c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6328073380 con 0x7f632809aa30 2024-04-25T00:56:35.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632d762640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f632809c080 0x7f63281412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:35.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632d762640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f632809c080 0x7f63281412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60832/0 (socket says 172.21.15.112:60832) 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.770+0000 7f632d762640 1 -- 172.21.15.112:0/3021338697 learned_addr learned my addr 172.21.15.112:0/3021338697 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632cf61640 1 --2- 172.21.15.112:0/3021338697 >> v2:172.21.15.187:3300/0 conn(0x7f632809f3e0 0x7f632809aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632d762640 1 -- 172.21.15.112:0/3021338697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 msgr2=0x7f63280bbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f6327fff640 1 --2- 172.21.15.112:0/3021338697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 0x7f63280bbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632d762640 1 --2- 172.21.15.112:0/3021338697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 0x7f63280bbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632d762640 1 -- 172.21.15.112:0/3021338697 >> v2:172.21.15.187:3300/0 conn(0x7f632809f3e0 msgr2=0x7f632809aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632d762640 1 --2- 172.21.15.112:0/3021338697 >> v2:172.21.15.187:3300/0 conn(0x7f632809f3e0 0x7f632809aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632d762640 1 -- 172.21.15.112:0/3021338697 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6328073660 con 0x7f632809c080 2024-04-25T00:56:35.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632cf61640 1 --2- 172.21.15.112:0/3021338697 >> v2:172.21.15.187:3300/0 conn(0x7f632809f3e0 0x7f632809aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632d762640 1 --2- 172.21.15.112:0/3021338697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809c080 0x7f63281412e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f63280bde60 tx=0x7f632001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad21a156414a63a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f63277fe640 1 -- 172.21.15.112:0/3021338697 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200a56d0 con 0x7f632809c080 2024-04-25T00:56:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f63277fe640 1 -- 172.21.15.112:0/3021338697 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63200a5870 con 0x7f632809c080 2024-04-25T00:56:35.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f63277fe640 1 -- 172.21.15.112:0/3021338697 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200a5c40 con 0x7f632809c080 2024-04-25T00:56:35.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 -- 172.21.15.112:0/3021338697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809c080 msgr2=0x7f63281412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:35.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 --2- 172.21.15.112:0/3021338697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809c080 0x7f63281412e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f63280bde60 tx=0x7f632001c960 comp rx=0 tx=0).stop 2024-04-25T00:56:35.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 -- 172.21.15.112:0/3021338697 shutdown_connections 2024-04-25T00:56:35.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 --2- 172.21.15.112:0/3021338697 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809c080 0x7f63281412e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 --2- 172.21.15.112:0/3021338697 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 0x7f63280bbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 --2- 172.21.15.112:0/3021338697 >> v2:172.21.15.187:3300/0 conn(0x7f632809f3e0 0x7f632809aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 -- 172.21.15.112:0/3021338697 >> 172.21.15.112:0/3021338697 conn(0x7f6328067550 msgr2=0x7f632809ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 -- 172.21.15.112:0/3021338697 shutdown_connections 2024-04-25T00:56:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.774+0000 7f632f1c5640 1 -- 172.21.15.112:0/3021338697 wait complete. 2024-04-25T00:56:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 Processor -- start 2024-04-25T00:56:35.779 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 -- start start 2024-04-25T00:56:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 0x7f63281e3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f632809c080 0x7f632813a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809f3e0 0x7f632809fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f632814b3f0 con 0x7f632809f3e0 2024-04-25T00:56:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f632814b270 con 0x7f632809c080 2024-04-25T00:56:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f632814b570 con 0x7f632809aa30 2024-04-25T00:56:35.782 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809f3e0 0x7f632809fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809f3e0 0x7f632809fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60846/0 (socket says 172.21.15.112:60846) 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 -- 172.21.15.112:0/730554404 learned_addr learned my addr 172.21.15.112:0/730554404 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632cf61640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 0x7f63281e3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f6327fff640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f632809c080 0x7f632813a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 -- 172.21.15.112:0/730554404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 msgr2=0x7f63281e3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 0x7f63281e3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 -- 172.21.15.112:0/730554404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f632809c080 msgr2=0x7f632813a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f632809c080 0x7f632813a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 -- 172.21.15.112:0/730554404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632813acf0 con 0x7f632809f3e0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632d762640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809f3e0 0x7f632809fb20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f63200af040 tx=0x7f6320019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f63257fa640 1 -- 172.21.15.112:0/730554404 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200bc020 con 0x7f632809f3e0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f63257fa640 1 -- 172.21.15.112:0/730554404 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63200b8670 con 0x7f632809f3e0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632813afd0 con 0x7f632809f3e0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f63257fa640 1 -- 172.21.15.112:0/730554404 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63200b8a40 con 0x7f632809f3e0 2024-04-25T00:56:35.783 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6328142b80 con 0x7f632809f3e0 2024-04-25T00:56:35.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6328068260 con 0x7f632809f3e0 2024-04-25T00:56:35.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.778+0000 7f63257fa640 1 -- 172.21.15.112:0/730554404 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f63200bf020 con 0x7f632809f3e0 2024-04-25T00:56:35.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.782+0000 7f63257fa640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f63140678c0 0x7f6314069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:35.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.782+0000 7f63257fa640 1 -- 172.21.15.112:0/730554404 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5357+0+0 (secure 0 0 0) 0x7f6320120c50 con 0x7f632809f3e0 2024-04-25T00:56:35.784 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.782+0000 7f632cf61640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f63140678c0 0x7f6314069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:35.785 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.782+0000 7f632cf61640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f63140678c0 0x7f6314069d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6318006fd0 tx=0x7f6318008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:35.787 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.786+0000 7f63257fa640 1 -- 172.21.15.112:0/730554404 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6320126020 con 0x7f632809f3e0 2024-04-25T00:56:35.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:35.946+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f63280a1d90 con 0x7f632809f3e0 2024-04-25T00:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.614+0000 7f63257fa640 1 -- 172.21.15.112:0/730554404 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f63200c6030 con 0x7f632809f3e0 2024-04-25T00:56:36.617 INFO:teuthology.orchestra.run.smithi112.stderr:pool 'cephfs_data' removed 2024-04-25T00:56:36.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f63140678c0 msgr2=0x7f6314069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:36.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f63140678c0 0x7f6314069d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6318006fd0 tx=0x7f6318008040 comp rx=0 tx=0).stop 2024-04-25T00:56:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809f3e0 msgr2=0x7f632809fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:36.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809f3e0 0x7f632809fb20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f63200af040 tx=0x7f6320019700 comp rx=0 tx=0).stop 2024-04-25T00:56:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 shutdown_connections 2024-04-25T00:56:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f632809f3e0 0x7f632809fb20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f63140678c0 0x7f6314069d80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f632809c080 0x7f632813a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 --2- 172.21.15.112:0/730554404 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f632809aa30 0x7f63281e3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.618+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 >> 172.21.15.112:0/730554404 conn(0x7f6328067550 msgr2=0x7f6328147cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.622+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 shutdown_connections 2024-04-25T00:56:36.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.622+0000 7f632f1c5640 1 -- 172.21.15.112:0/730554404 wait complete. 2024-04-25T00:56:36.644 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T00:56:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16ae5cb640 1 Processor -- start 2024-04-25T00:56:36.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16ae5cb640 1 -- start start 2024-04-25T00:56:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16ae5cb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16a806c170 0x7f16a80aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16ae5cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 0x7f16a80b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16ae5cb640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f16a80b3e50 0x7f16a8147900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:36.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16ae5cb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f16a80ae8a0 con 0x7f16a806c170 2024-04-25T00:56:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16ae5cb640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f16a80b42e0 con 0x7f16a80b3e50 2024-04-25T00:56:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16ae5cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16a806c550 con 0x7f16a80b34b0 2024-04-25T00:56:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.762+0000 7f16a7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16a806c170 0x7f16a80aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f16a806c170 0x7f16a80aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60852/0 (socket says 172.21.15.112:60852) 2024-04-25T00:56:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a7fff640 1 -- 172.21.15.112:0/206389972 learned_addr learned my addr 172.21.15.112:0/206389972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16acb68640 1 --2- 172.21.15.112:0/206389972 >> v2:172.21.15.187:3300/0 conn(0x7f16a80b3e50 0x7f16a8147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:36.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a7fff640 1 -- 172.21.15.112:0/206389972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 msgr2=0x7f16a80b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:36.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a7fff640 1 --2- 172.21.15.112:0/206389972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 0x7f16a80b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a7fff640 1 -- 172.21.15.112:0/206389972 >> v2:172.21.15.187:3300/0 conn(0x7f16a80b3e50 msgr2=0x7f16a8147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:36.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a7fff640 1 --2- 172.21.15.112:0/206389972 >> v2:172.21.15.187:3300/0 conn(0x7f16a80b3e50 0x7f16a8147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a7fff640 1 -- 172.21.15.112:0/206389972 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a80ae220 con 0x7f16a806c170 2024-04-25T00:56:36.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a7fff640 1 --2- 172.21.15.112:0/206389972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a806c170 0x7f16a80aedd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1698009870 tx=0x7f169801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f43687b67f5c4b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:36.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a6ffd640 1 -- 172.21.15.112:0/206389972 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16980b0070 con 0x7f16a806c170 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a6ffd640 1 -- 172.21.15.112:0/206389972 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16980ab600 con 0x7f16a806c170 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16a6ffd640 1 -- 172.21.15.112:0/206389972 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16980ab9d0 con 0x7f16a806c170 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 -- 172.21.15.112:0/206389972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a806c170 msgr2=0x7f16a80aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/206389972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a806c170 0x7f16a80aedd0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1698009870 tx=0x7f169801c1d0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 -- 172.21.15.112:0/206389972 shutdown_connections 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/206389972 >> v2:172.21.15.187:3300/0 conn(0x7f16a80b3e50 0x7f16a8147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/206389972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 0x7f16a80b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/206389972 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a806c170 0x7f16a80aedd0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 -- 172.21.15.112:0/206389972 >> 172.21.15.112:0/206389972 conn(0x7f16a80b8c80 msgr2=0x7f16a80b9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 -- 172.21.15.112:0/206389972 shutdown_connections 2024-04-25T00:56:36.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.766+0000 7f16ae5cb640 1 -- 172.21.15.112:0/206389972 wait complete. 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 Processor -- start 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 -- start start 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16a806c170 0x7f16a81e13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 0x7f16a81435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a80b3e50 0x7f16a813fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f16a80b2ea0 con 0x7f16a80b3e50 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f16a80adef0 con 0x7f16a806c170 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f16a80b51f0 con 0x7f16a80b34b0 2024-04-25T00:56:36.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a80b3e50 0x7f16a813fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a80b3e50 0x7f16a813fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60866/0 (socket says 172.21.15.112:60866) 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 -- 172.21.15.112:0/1626186189 learned_addr learned my addr 172.21.15.112:0/1626186189 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a7fff640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16a806c170 0x7f16a81e13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a77fe640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 0x7f16a81435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 -- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 msgr2=0x7f16a81435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 0x7f16a81435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 -- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16a806c170 msgr2=0x7f16a81e13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16a806c170 0x7f16a81e13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 -- 172.21.15.112:0/1626186189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a80afef0 con 0x7f16a80b3e50 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a7fff640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16a806c170 0x7f16a81e13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16acb68640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a80b3e50 0x7f16a813fb30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f16a0007e40 tx=0x7f16a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:36.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a4ff9640 1 -- 172.21.15.112:0/1626186189 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0014620 con 0x7f16a80b3e50 2024-04-25T00:56:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a4ff9640 1 -- 172.21.15.112:0/1626186189 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a0019070 con 0x7f16a80b3e50 2024-04-25T00:56:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a8140b70 con 0x7f16a80b3e50 2024-04-25T00:56:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a4ff9640 1 -- 172.21.15.112:0/1626186189 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0014a20 con 0x7f16a80b3e50 2024-04-25T00:56:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a813cca0 con 0x7f16a80b3e50 2024-04-25T00:56:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a804f5f0 con 0x7f16a80b3e50 2024-04-25T00:56:36.777 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a4ff9640 1 -- 172.21.15.112:0/1626186189 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f16a0014bc0 con 0x7f16a80b3e50 2024-04-25T00:56:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a4ff9640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16940678c0 0x7f1694069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a4ff9640 1 -- 172.21.15.112:0/1626186189 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f16a0086210 con 0x7f16a80b3e50 2024-04-25T00:56:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.770+0000 7f16a7fff640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16940678c0 0x7f1694069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.774+0000 7f16a7fff640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16940678c0 0x7f1694069d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16980ad000 tx=0x7f1698008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:36.778 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.778+0000 7f16a4ff9640 1 -- 172.21.15.112:0/1626186189 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f16a00570e0 con 0x7f16a80b3e50 2024-04-25T00:56:36.941 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.938+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f16a8140d40 con 0x7f16a80b3e50 2024-04-25T00:56:36.945 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.942+0000 7f16a4ff9640 1 -- 172.21.15.112:0/1626186189 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f16a00569e0 con 0x7f16a80b3e50 2024-04-25T00:56:36.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16940678c0 msgr2=0x7f1694069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:36.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16940678c0 0x7f1694069d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f16980ad000 tx=0x7f1698008040 comp rx=0 tx=0).stop 2024-04-25T00:56:36.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a80b3e50 msgr2=0x7f16a813fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:36.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a80b3e50 0x7f16a813fb30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f16a0007e40 tx=0x7f16a000e4b0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.948 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 shutdown_connections 2024-04-25T00:56:36.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f16a80b3e50 0x7f16a813fb30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f16a80b34b0 0x7f16a81435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.949 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f16940678c0 0x7f1694069d80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 --2- 172.21.15.112:0/1626186189 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f16a806c170 0x7f16a81e13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 >> 172.21.15.112:0/1626186189 conn(0x7f16a80b8c80 msgr2=0x7f16a80b70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 shutdown_connections 2024-04-25T00:56:36.950 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:36.946+0000 7f16ae5cb640 1 -- 172.21.15.112:0/1626186189 wait complete. 2024-04-25T00:56:36.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-04-25T00:56:37.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f1414831640 1 Processor -- start 2024-04-25T00:56:37.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f1414831640 1 -- start start 2024-04-25T00:56:37.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f1414831640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100add70 0x7f1410145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:37.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f1414831640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f14100b3ce0 0x7f141014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:37.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f1414831640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14100aeec0 con 0x7f14100b23f0 2024-04-25T00:56:37.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f1414831640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f14100af780 con 0x7f14100b3ce0 2024-04-25T00:56:37.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f1414831640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14100b4170 con 0x7f14100add70 2024-04-25T00:56:37.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f140dd9b640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100add70 0x7f1410145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:37.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f140ed9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f14100b3ce0 0x7f141014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:37.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f140ed9d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f14100b3ce0 0x7f141014fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40066/0 (socket says 172.21.15.112:40066) 2024-04-25T00:56:37.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f140ed9d640 1 -- 172.21.15.112:0/4200706321 learned_addr learned my addr 172.21.15.112:0/4200706321 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:37.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f140ed9d640 1 -- 172.21.15.112:0/4200706321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100add70 msgr2=0x7f1410145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:37.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f140ed9d640 1 --2- 172.21.15.112:0/4200706321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100add70 0x7f1410145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f140ed9d640 1 -- 172.21.15.112:0/4200706321 >> v1:172.21.15.187:6789/0 conn(0x7f14100b23f0 legacy=0x7f1410072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:37.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.086+0000 7f140ed9d640 1 -- 172.21.15.112:0/4200706321 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14100b27d0 con 0x7f14100b3ce0 2024-04-25T00:56:37.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f140ed9d640 1 --2- 172.21.15.112:0/4200706321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 0x7f141014fb40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1408009e50 tx=0x7f140801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7406af15f5889d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:37.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f140d59a640 1 -- 172.21.15.112:0/4200706321 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080a5810 con 0x7f14100b3ce0 2024-04-25T00:56:37.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f140d59a640 1 -- 172.21.15.112:0/4200706321 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14080a59b0 con 0x7f14100b3ce0 2024-04-25T00:56:37.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f140d59a640 1 -- 172.21.15.112:0/4200706321 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14080a5d80 con 0x7f14100b3ce0 2024-04-25T00:56:37.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- 172.21.15.112:0/4200706321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 msgr2=0x7f141014fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:37.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 --2- 172.21.15.112:0/4200706321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 0x7f141014fb40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1408009e50 tx=0x7f140801ca30 comp rx=0 tx=0).stop 2024-04-25T00:56:37.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- 172.21.15.112:0/4200706321 shutdown_connections 2024-04-25T00:56:37.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 --2- 172.21.15.112:0/4200706321 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 0x7f141014fb40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 --2- 172.21.15.112:0/4200706321 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100add70 0x7f1410145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- 172.21.15.112:0/4200706321 >> 172.21.15.112:0/4200706321 conn(0x7f14100b5980 msgr2=0x7f14100b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:37.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- 172.21.15.112:0/4200706321 shutdown_connections 2024-04-25T00:56:37.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- 172.21.15.112:0/4200706321 wait complete. 2024-04-25T00:56:37.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 Processor -- start 2024-04-25T00:56:37.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- start start 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14100add70 0x7f1410130800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100b23f0 0x7f1410146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 0x7f1410147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f14100b52d0 con 0x7f14100add70 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f14100b5150 con 0x7f14100b3ce0 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.090+0000 7f1414831640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14101528a0 con 0x7f14100b23f0 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14100add70 0x7f1410130800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14100add70 0x7f1410130800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60878/0 (socket says 172.21.15.112:60878) 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 -- 172.21.15.112:0/2780413940 learned_addr learned my addr 172.21.15.112:0/2780413940 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140ed9d640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 0x7f1410147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140dd9b640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100b23f0 0x7f1410146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 -- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100b23f0 msgr2=0x7f1410146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:37.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100b23f0 0x7f1410146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 -- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 msgr2=0x7f1410147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 0x7f1410147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 -- 172.21.15.112:0/2780413940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14100720d0 con 0x7f14100add70 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140ed9d640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 0x7f1410147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140e59c640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14100add70 0x7f1410130800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f140400c420 tx=0x7f140400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f13feffd640 1 -- 172.21.15.112:0/2780413940 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14040052d0 con 0x7f14100add70 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f13feffd640 1 -- 172.21.15.112:0/2780413940 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1404005470 con 0x7f14100add70 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f13feffd640 1 -- 172.21.15.112:0/2780413940 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404007260 con 0x7f14100add70 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14101e2260 con 0x7f14100add70 2024-04-25T00:56:37.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1410146920 con 0x7f14100add70 2024-04-25T00:56:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141004f580 con 0x7f14100add70 2024-04-25T00:56:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f13feffd640 1 -- 172.21.15.112:0/2780413940 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1404007400 con 0x7f14100add70 2024-04-25T00:56:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f13feffd640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f13f00678f0 0x7f13f0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f13feffd640 1 -- 172.21.15.112:0/2780413940 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f1404084df0 con 0x7f14100add70 2024-04-25T00:56:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140dd9b640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f13f00678f0 0x7f13f0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:37.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.094+0000 7f140dd9b640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f13f00678f0 0x7f13f0069db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f14101dbb10 tx=0x7f1400008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:37.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.098+0000 7f13feffd640 1 -- 172.21.15.112:0/2780413940 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1404055cc0 con 0x7f14100add70 2024-04-25T00:56:37.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.322+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7f1410146af0 con 0x7f14100add70 2024-04-25T00:56:37.643 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f13feffd640 1 -- 172.21.15.112:0/2780413940 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v79) ==== 134+0+0 (secure 0 0 0) 0x7f14040555c0 con 0x7f14100add70 2024-04-25T00:56:37.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f13f00678f0 msgr2=0x7f13f0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:37.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f13f00678f0 0x7f13f0069db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f14101dbb10 tx=0x7f1400008040 comp rx=0 tx=0).stop 2024-04-25T00:56:37.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14100add70 msgr2=0x7f1410130800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:37.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14100add70 0x7f1410130800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f140400c420 tx=0x7f140400c8e0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 shutdown_connections 2024-04-25T00:56:37.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f13f00678f0 0x7f13f0069db0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f14100b3ce0 0x7f1410147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14100b23f0 0x7f1410146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 --2- 172.21.15.112:0/2780413940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14100add70 0x7f1410130800 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:37.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 >> 172.21.15.112:0/2780413940 conn(0x7f14100b5980 msgr2=0x7f141006c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:37.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 shutdown_connections 2024-04-25T00:56:37.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:37.642+0000 7f1414831640 1 -- 172.21.15.112:0/2780413940 wait complete. 2024-04-25T00:56:37.669 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed ... ERROR 2024-04-25T00:56:37.670 INFO:tasks.cephfs_test_runner: 2024-04-25T00:56:37.670 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T00:56:37.670 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-25T00:56:37.670 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-04-25T00:56:37.670 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T00:56:37.670 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T00:56:37.670 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-04-25T00:56:37.670 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-25T00:56:37.671 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-25T00:56:37.672 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi112 with status 127: 'cephfs-shell -b /tmp/test-cephfsgw7smiww' 2024-04-25T00:56:37.672 INFO:tasks.cephfs_test_runner: 2024-04-25T00:56:37.672 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T00:56:37.672 INFO:tasks.cephfs_test_runner:Ran 1 test in 38.475s 2024-04-25T00:56:37.672 INFO:tasks.cephfs_test_runner: 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner: 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-04-25T00:56:37.673 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-04-25T00:56:37.674 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-04-25T00:56:37.674 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-04-25T00:56:37.674 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-04-25T00:56:37.674 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run 2024-04-25T00:56:37.674 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-04-25T00:56:37.674 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run 2024-04-25T00:56:37.674 INFO:tasks.cephfs_test_runner: r.wait() 2024-04-25T00:56:37.675 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait 2024-04-25T00:56:37.675 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-04-25T00:56:37.675 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-04-25T00:56:37.675 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-04-25T00:56:37.675 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi112 with status 127: 'cephfs-shell -b /tmp/test-cephfsgw7smiww' 2024-04-25T00:56:37.675 INFO:tasks.cephfs_test_runner: 2024-04-25T00:56:37.675 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-25T00:56:37.931 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=127d8d3c36ea41b380b54405929e021b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-04-25T00:56:37.932 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T00:56:37.948 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T00:56:37.966 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T00:56:37.967 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2024-04-25T00:56:37.967 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:56:37.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16151 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-04-25T00:56:37.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:37.981 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2024-04-25T00:56:38.028 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:56:38.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:38.029 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:56:38.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:56:38.085 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2024-04-25T00:56:38.293 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T00:56:38.306 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T00:56:38.306 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:56:38.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.430+0000 7f0651181640 1 Processor -- start 2024-04-25T00:56:38.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.430+0000 7f0651181640 1 -- start start 2024-04-25T00:56:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f0651181640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f064c06ca10 0x7f064c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f0651181640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f064c0add70 0x7f064c0b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f0651181640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c0af760 con 0x7f064c06c630 2024-04-25T00:56:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f0651181640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f064c148860 con 0x7f064c06ca10 2024-04-25T00:56:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f0651181640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f064c0b52f0 con 0x7f064c0add70 2024-04-25T00:56:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f064c06ca10 0x7f064c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064a59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f064c06ca10 0x7f064c0aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60892/0 (socket says 172.21.15.112:60892) 2024-04-25T00:56:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064a59c640 1 -- 172.21.15.112:0/3284431908 learned_addr learned my addr 172.21.15.112:0/3284431908 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064b59e640 1 --2- 172.21.15.112:0/3284431908 >> v2:172.21.15.187:3300/0 conn(0x7f064c0add70 0x7f064c0b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064a59c640 1 -- 172.21.15.112:0/3284431908 >> v2:172.21.15.187:3300/0 conn(0x7f064c0add70 msgr2=0x7f064c0b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064a59c640 1 --2- 172.21.15.112:0/3284431908 >> v2:172.21.15.187:3300/0 conn(0x7f064c0add70 0x7f064c0b4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064a59c640 1 -- 172.21.15.112:0/3284431908 >> v1:172.21.15.187:6789/0 conn(0x7f064c06c630 legacy=0x7f064c1477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:38.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064a59c640 1 -- 172.21.15.112:0/3284431908 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c0afa00 con 0x7f064c06ca10 2024-04-25T00:56:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f064a59c640 1 --2- 172.21.15.112:0/3284431908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c06ca10 0x7f064c0aff80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f064c0b04c0 tx=0x7f064001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f40d8269792675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f0649d9b640 1 -- 172.21.15.112:0/3284431908 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400b0070 con 0x7f064c06ca10 2024-04-25T00:56:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f0649d9b640 1 -- 172.21.15.112:0/3284431908 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06400ab600 con 0x7f064c06ca10 2024-04-25T00:56:38.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.434+0000 7f0649d9b640 1 -- 172.21.15.112:0/3284431908 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400ab9d0 con 0x7f064c06ca10 2024-04-25T00:56:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- 172.21.15.112:0/3284431908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c06ca10 msgr2=0x7f064c0aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 --2- 172.21.15.112:0/3284431908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c06ca10 0x7f064c0aff80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f064c0b04c0 tx=0x7f064001c3a0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- 172.21.15.112:0/3284431908 shutdown_connections 2024-04-25T00:56:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 --2- 172.21.15.112:0/3284431908 >> v2:172.21.15.187:3300/0 conn(0x7f064c0add70 0x7f064c0b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 --2- 172.21.15.112:0/3284431908 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c06ca10 0x7f064c0aff80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- 172.21.15.112:0/3284431908 >> 172.21.15.112:0/3284431908 conn(0x7f064c0b5980 msgr2=0x7f064c0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:38.440 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- 172.21.15.112:0/3284431908 shutdown_connections 2024-04-25T00:56:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- 172.21.15.112:0/3284431908 wait complete. 2024-04-25T00:56:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 Processor -- start 2024-04-25T00:56:38.441 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- start start 2024-04-25T00:56:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f064c06c630 0x7f064c146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f064c06ca10 0x7f064c09ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c0add70 0x7f064c09b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f064c1519f0 con 0x7f064c0add70 2024-04-25T00:56:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f064c151870 con 0x7f064c06c630 2024-04-25T00:56:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f064c151b70 con 0x7f064c06ca10 2024-04-25T00:56:38.443 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c0add70 0x7f064c09b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c0add70 0x7f064c09b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60906/0 (socket says 172.21.15.112:60906) 2024-04-25T00:56:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 -- 172.21.15.112:0/3876576850 learned_addr learned my addr 172.21.15.112:0/3876576850 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 -- 172.21.15.112:0/3876576850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f064c06ca10 msgr2=0x7f064c09ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f064c06ca10 0x7f064c09ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 -- 172.21.15.112:0/3876576850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f064c06c630 msgr2=0x7f064c146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064ad9d640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f064c06c630 0x7f064c146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f064c06c630 0x7f064c146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 -- 172.21.15.112:0/3876576850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c143810 con 0x7f064c0add70 2024-04-25T00:56:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064ad9d640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f064c06c630 0x7f064c146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f064b59e640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c0add70 0x7f064c09b2a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0644007ca0 tx=0x7f064400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f06337fe640 1 -- 172.21.15.112:0/3876576850 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644014420 con 0x7f064c0add70 2024-04-25T00:56:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f06337fe640 1 -- 172.21.15.112:0/3876576850 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0644019070 con 0x7f064c0add70 2024-04-25T00:56:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c143530 con 0x7f064c0add70 2024-04-25T00:56:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f06337fe640 1 -- 172.21.15.112:0/3876576850 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0644014800 con 0x7f064c0add70 2024-04-25T00:56:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.438+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c14eb70 con 0x7f064c0add70 2024-04-25T00:56:38.445 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.442+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f064c143320 con 0x7f064c0add70 2024-04-25T00:56:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.442+0000 7f06337fe640 1 -- 172.21.15.112:0/3876576850 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06440149a0 con 0x7f064c0add70 2024-04-25T00:56:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.442+0000 7f06337fe640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06380678c0 0x7f0638069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.442+0000 7f06337fe640 1 -- 172.21.15.112:0/3876576850 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f06440860b0 con 0x7f064c0add70 2024-04-25T00:56:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.442+0000 7f064ad9d640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06380678c0 0x7f0638069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.447 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.442+0000 7f064ad9d640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06380678c0 0x7f0638069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f063c003e40 tx=0x7f063c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:38.448 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.446+0000 7f06337fe640 1 -- 172.21.15.112:0/3876576850 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0644056f80 con 0x7f064c0add70 2024-04-25T00:56:38.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.602+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f064c04f580 con 0x7f06380678c0 2024-04-25T00:56:38.606 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.602+0000 7f06337fe640 1 -- 172.21.15.112:0/3876576850 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13373 (secure 0 0 0) 0x7f063c01b450 con 0x7f06380678c0 2024-04-25T00:56:38.606 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:38.607 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:56:38.608 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06380678c0 msgr2=0x7f0638069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06380678c0 0x7f0638069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f063c003e40 tx=0x7f063c009040 comp rx=0 tx=0).stop 2024-04-25T00:56:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c0add70 msgr2=0x7f064c09b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c0add70 0x7f064c09b2a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0644007ca0 tx=0x7f064400e4b0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 shutdown_connections 2024-04-25T00:56:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f064c0add70 0x7f064c09b2a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f064c06ca10 0x7f064c09ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f06380678c0 0x7f0638069d80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 --2- 172.21.15.112:0/3876576850 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f064c06c630 0x7f064c146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 >> 172.21.15.112:0/3876576850 conn(0x7f064c0b5980 msgr2=0x7f064c0ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:38.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 shutdown_connections 2024-04-25T00:56:38.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.606+0000 7f0651181640 1 -- 172.21.15.112:0/3876576850 wait complete. 2024-04-25T00:56:38.625 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-04-25T00:56:37.530261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224096,"kb_used_data":3768,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750624,"statfs":{"total":773094113280,"available":772864638976,"internally_reserved":0,"allocated":3858432,"data_stored":2194614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2851,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001378"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025163+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:56:21.025163+0000","last_peered":"2024-04-25T00:56:21.025163+0000","last_clean":"2024-04-25T00:56:21.025163+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:56:21.025163+0000","last_undegraded":"2024-04-25T00:56:21.025163+0000","last_fullsized":"2024-04-25T00:56:21.025163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":124082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":917504,"data_stored":714380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:56:38.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:56:38.747 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b6c71640 1 Processor -- start 2024-04-25T00:56:38.890 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b6c71640 1 -- start start 2024-04-25T00:56:38.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b6c71640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b47f0 0x7fa0b0147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b6c71640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fa0b0148400 0x7fa0b00b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b6c71640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0b00b2640 0x7fa0b006c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b6c71640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b00aff80 con 0x7fa0b00b2640 2024-04-25T00:56:38.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b6c71640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b00b1c70 con 0x7fa0b0148400 2024-04-25T00:56:38.896 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b6c71640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b00b1060 con 0x7fa0b00b47f0 2024-04-25T00:56:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0b00b2640 0x7fa0b006c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa0b00b2640 0x7fa0b006c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60910/0 (socket says 172.21.15.112:60910) 2024-04-25T00:56:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 -- 172.21.15.112:0/3291136182 learned_addr learned my addr 172.21.15.112:0/3291136182 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0affff640 1 --2- 172.21.15.112:0/3291136182 >> v2:172.21.15.187:3300/0 conn(0x7fa0b0148400 0x7fa0b00b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 -- 172.21.15.112:0/3291136182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b47f0 msgr2=0x7fa0b0147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 --2- 172.21.15.112:0/3291136182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b47f0 0x7fa0b0147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 -- 172.21.15.112:0/3291136182 >> v2:172.21.15.187:3300/0 conn(0x7fa0b0148400 msgr2=0x7fa0b00b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:38.897 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 --2- 172.21.15.112:0/3291136182 >> v2:172.21.15.187:3300/0 conn(0x7fa0b0148400 0x7fa0b00b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 -- 172.21.15.112:0/3291136182 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b00b2ad0 con 0x7fa0b00b2640 2024-04-25T00:56:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0affff640 1 --2- 172.21.15.112:0/3291136182 >> v2:172.21.15.187:3300/0 conn(0x7fa0b0148400 0x7fa0b00b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0b520e640 1 --2- 172.21.15.112:0/3291136182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b00b2640 0x7fa0b006c2c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa0b006da20 tx=0x7fa0a801e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51eeb7b25702cbbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0af7fe640 1 -- 172.21.15.112:0/3291136182 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800b2d0 con 0x7fa0b00b2640 2024-04-25T00:56:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0af7fe640 1 -- 172.21.15.112:0/3291136182 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a800b470 con 0x7fa0b00b2640 2024-04-25T00:56:38.898 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.746+0000 7fa0af7fe640 1 -- 172.21.15.112:0/3291136182 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8009820 con 0x7fa0b00b2640 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- 172.21.15.112:0/3291136182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b00b2640 msgr2=0x7fa0b006c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/3291136182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b00b2640 0x7fa0b006c2c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa0b006da20 tx=0x7fa0a801e8a0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- 172.21.15.112:0/3291136182 shutdown_connections 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/3291136182 >> v2:172.21.15.187:3300/0 conn(0x7fa0b0148400 0x7fa0b00b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/3291136182 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b00b2640 0x7fa0b006c2c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/3291136182 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b47f0 0x7fa0b0147ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- 172.21.15.112:0/3291136182 >> 172.21.15.112:0/3291136182 conn(0x7fa0b00b5980 msgr2=0x7fa0b00b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- 172.21.15.112:0/3291136182 shutdown_connections 2024-04-25T00:56:38.899 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- 172.21.15.112:0/3291136182 wait complete. 2024-04-25T00:56:38.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 Processor -- start 2024-04-25T00:56:38.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- start start 2024-04-25T00:56:38.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b2640 0x7fa0b013dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00b47f0 0x7fa0b01df7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b0148400 0x7fa0b00b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b00b1900 con 0x7fa0b0148400 2024-04-25T00:56:38.900 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b00b01d0 con 0x7fa0b00b47f0 2024-04-25T00:56:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b00ae280 con 0x7fa0b00b2640 2024-04-25T00:56:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b0148400 0x7fa0b00b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b0148400 0x7fa0b00b8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60912/0 (socket says 172.21.15.112:60912) 2024-04-25T00:56:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 -- 172.21.15.112:0/1929737954 learned_addr learned my addr 172.21.15.112:0/1929737954 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0affff640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00b47f0 0x7fa0b01df7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b4a0d640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b2640 0x7fa0b013dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.901 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 -- 172.21.15.112:0/1929737954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b2640 msgr2=0x7fa0b013dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b2640 0x7fa0b013dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 -- 172.21.15.112:0/1929737954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00b47f0 msgr2=0x7fa0b01df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00b47f0 0x7fa0b01df7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 -- 172.21.15.112:0/1929737954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b013cbf0 con 0x7fa0b0148400 2024-04-25T00:56:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b520e640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b0148400 0x7fa0b00b8e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80b1040 tx=0x7fa0a801b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0ad7fa640 1 -- 172.21.15.112:0/1929737954 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80be030 con 0x7fa0b0148400 2024-04-25T00:56:38.902 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0ad7fa640 1 -- 172.21.15.112:0/1929737954 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a80a7530 con 0x7fa0b0148400 2024-04-25T00:56:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b00b7650 con 0x7fa0b0148400 2024-04-25T00:56:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0ad7fa640 1 -- 172.21.15.112:0/1929737954 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80a7900 con 0x7fa0b0148400 2024-04-25T00:56:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.750+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b013e010 con 0x7fa0b0148400 2024-04-25T00:56:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.754+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b004f580 con 0x7fa0b0148400 2024-04-25T00:56:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.754+0000 7fa0ad7fa640 1 -- 172.21.15.112:0/1929737954 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa0a80c1070 con 0x7fa0b0148400 2024-04-25T00:56:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.754+0000 7fa0ad7fa640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa09c0678c0 0x7fa09c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.754+0000 7fa0ad7fa640 1 -- 172.21.15.112:0/1929737954 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fa0a8122740 con 0x7fa0b0148400 2024-04-25T00:56:38.903 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.754+0000 7fa0b4a0d640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa09c0678c0 0x7fa09c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.754+0000 7fa0b4a0d640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa09c0678c0 0x7fa09c069d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0006fd0 tx=0x7fa0a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:38.904 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.758+0000 7fa0ad7fa640 1 -- 172.21.15.112:0/1929737954 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa0a80c8070 con 0x7fa0b0148400 2024-04-25T00:56:38.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.914+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa0b006fa10 con 0x7fa09c0678c0 2024-04-25T00:56:38.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.914+0000 7fa0ad7fa640 1 -- 172.21.15.112:0/1929737954 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13373 (secure 0 0 0) 0x7fa0b006fa10 con 0x7fa09c0678c0 2024-04-25T00:56:38.928 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:38.928 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:56:38.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa09c0678c0 msgr2=0x7fa09c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa09c0678c0 0x7fa09c069d80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0006fd0 tx=0x7fa0a0008040 comp rx=0 tx=0).stop 2024-04-25T00:56:38.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b0148400 msgr2=0x7fa0b00b8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:38.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b0148400 0x7fa0b00b8e60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa0a80b1040 tx=0x7fa0a801b640 comp rx=0 tx=0).stop 2024-04-25T00:56:38.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 shutdown_connections 2024-04-25T00:56:38.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa0b0148400 0x7fa0b00b8e60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa09c0678c0 0x7fa09c069d80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa0b00b47f0 0x7fa0b01df7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 --2- 172.21.15.112:0/1929737954 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa0b00b2640 0x7fa0b013dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:38.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 >> 172.21.15.112:0/1929737954 conn(0x7fa0b00b5980 msgr2=0x7fa0b01414c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:38.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 shutdown_connections 2024-04-25T00:56:38.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:38.918+0000 7fa0b6c71640 1 -- 172.21.15.112:0/1929737954 wait complete. 2024-04-25T00:56:38.947 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-04-25T00:56:37.530261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224096,"kb_used_data":3768,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750624,"statfs":{"total":773094113280,"available":772864638976,"internally_reserved":0,"allocated":3858432,"data_stored":2194614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2851,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001378"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025163+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:56:21.025163+0000","last_peered":"2024-04-25T00:56:21.025163+0000","last_clean":"2024-04-25T00:56:21.025163+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:56:21.025163+0000","last_undegraded":"2024-04-25T00:56:21.025163+0000","last_fullsized":"2024-04-25T00:56:21.025163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":124082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":917504,"data_stored":714380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:56:38.948 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T00:56:38.949 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:56:39.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:56:39.054 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:56:39.087 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.082+0000 7f526e9cf640 1 Processor -- start 2024-04-25T00:56:39.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.082+0000 7f526e9cf640 1 -- start start 2024-04-25T00:56:39.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f52680b23f0 0x7f52680add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52680aeec0 con 0x7f5268145200 2024-04-25T00:56:39.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52680af780 con 0x7f52680b3ce0 2024-04-25T00:56:39.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f52680b27d0 con 0x7f52680b23f0 2024-04-25T00:56:39.088 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f5267fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f52680b23f0 0x7f52680add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f52677fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f5268145200 0x7f5268072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32818/0 (socket says 172.21.15.112:32818) 2024-04-25T00:56:39.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f52677fe640 1 -- 172.21.15.112:0/21641607 learned_addr learned my addr 172.21.15.112:0/21641607 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:39.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f5267fff640 1 -- 172.21.15.112:0/21641607 >> v1:172.21.15.187:6789/0 conn(0x7f52680b3ce0 legacy=0x7f526814fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:39.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f5267fff640 1 -- 172.21.15.112:0/21641607 >> v1:172.21.15.112:6789/0 conn(0x7f5268145200 legacy=0x7f5268072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.089 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f5267fff640 1 -- 172.21.15.112:0/21641607 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52680afa20 con 0x7f52680b23f0 2024-04-25T00:56:39.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f5267fff640 1 --2- 172.21.15.112:0/21641607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b23f0 0x7f52680add70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52680af950 tx=0x7f525c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24e628aa7181f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:39.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f5266ffd640 1 -- 172.21.15.112:0/21641607 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c0b0070 con 0x7f52680b23f0 2024-04-25T00:56:39.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f5266ffd640 1 -- 172.21.15.112:0/21641607 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f525c0ab600 con 0x7f52680b23f0 2024-04-25T00:56:39.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f5266ffd640 1 -- 172.21.15.112:0/21641607 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c0ab9d0 con 0x7f52680b23f0 2024-04-25T00:56:39.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 -- 172.21.15.112:0/21641607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b23f0 msgr2=0x7f52680add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 --2- 172.21.15.112:0/21641607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b23f0 0x7f52680add70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f52680af950 tx=0x7f525c01c460 comp rx=0 tx=0).stop 2024-04-25T00:56:39.092 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 -- 172.21.15.112:0/21641607 shutdown_connections 2024-04-25T00:56:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 --2- 172.21.15.112:0/21641607 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b23f0 0x7f52680add70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 -- 172.21.15.112:0/21641607 >> 172.21.15.112:0/21641607 conn(0x7f52680b5980 msgr2=0x7f52680b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 -- 172.21.15.112:0/21641607 shutdown_connections 2024-04-25T00:56:39.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.086+0000 7f526e9cf640 1 -- 172.21.15.112:0/21641607 wait complete. 2024-04-25T00:56:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 Processor -- start 2024-04-25T00:56:39.096 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 -- start start 2024-04-25T00:56:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52680b23f0 0x7f526813cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b3ce0 0x7f52681e0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5268145200 0x7f52681e08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52680b52d0 con 0x7f52680b3ce0 2024-04-25T00:56:39.097 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52680b5150 con 0x7f52680b23f0 2024-04-25T00:56:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f5268153a80 con 0x7f5268145200 2024-04-25T00:56:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b3ce0 0x7f52681e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b3ce0 0x7f52681e0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60926/0 (socket says 172.21.15.112:60926) 2024-04-25T00:56:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 -- 172.21.15.112:0/3976235153 learned_addr learned my addr 172.21.15.112:0/3976235153 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:39.098 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f5267fff640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52680b23f0 0x7f526813cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526cf6c640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5268145200 0x7f52681e08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 -- 172.21.15.112:0/3976235153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5268145200 msgr2=0x7f52681e08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5268145200 0x7f52681e08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 -- 172.21.15.112:0/3976235153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52680b23f0 msgr2=0x7f526813cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.099 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52680b23f0 0x7f526813cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 -- 172.21.15.112:0/3976235153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526813f2a0 con 0x7f52680b3ce0 2024-04-25T00:56:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f52677fe640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b3ce0 0x7f52681e0380 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f525800c370 tx=0x7f525800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f5264ff9640 1 -- 172.21.15.112:0/3976235153 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258017070 con 0x7f52680b3ce0 2024-04-25T00:56:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f5264ff9640 1 -- 172.21.15.112:0/3976235153 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5258005cf0 con 0x7f52680b3ce0 2024-04-25T00:56:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f5264ff9640 1 -- 172.21.15.112:0/3976235153 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580072f0 con 0x7f52680b3ce0 2024-04-25T00:56:39.100 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526813dfb0 con 0x7f52680b3ce0 2024-04-25T00:56:39.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526813f510 con 0x7f52680b3ce0 2024-04-25T00:56:39.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.090+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526804f580 con 0x7f52680b3ce0 2024-04-25T00:56:39.101 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.094+0000 7f5264ff9640 1 -- 172.21.15.112:0/3976235153 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f525801a070 con 0x7f52680b3ce0 2024-04-25T00:56:39.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.094+0000 7f5264ff9640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52540678f0 0x7f5254069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.094+0000 7f5264ff9640 1 -- 172.21.15.112:0/3976235153 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f5258084bb0 con 0x7f52680b3ce0 2024-04-25T00:56:39.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.094+0000 7f5267fff640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52540678f0 0x7f5254069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.094+0000 7f5267fff640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52540678f0 0x7f5254069db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f525c0ad000 tx=0x7f525c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:39.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.098+0000 7f5264ff9640 1 -- 172.21.15.112:0/3976235153 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5258055430 con 0x7f52680b3ce0 2024-04-25T00:56:39.255 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.254+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52680b8910 con 0x7f52540678f0 2024-04-25T00:56:39.256 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.254+0000 7f5264ff9640 1 -- 172.21.15.112:0/3976235153 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13373 (secure 0 0 0) 0x7f52680b8910 con 0x7f52540678f0 2024-04-25T00:56:39.257 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:39.258 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:56:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52540678f0 msgr2=0x7f5254069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52540678f0 0x7f5254069db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f525c0ad000 tx=0x7f525c0a6000 comp rx=0 tx=0).stop 2024-04-25T00:56:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b3ce0 msgr2=0x7f52681e0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.259 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b3ce0 0x7f52681e0380 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f525800c370 tx=0x7f525800c830 comp rx=0 tx=0).stop 2024-04-25T00:56:39.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 shutdown_connections 2024-04-25T00:56:39.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f5268145200 0x7f52681e08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52680b3ce0 0x7f52681e0380 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52540678f0 0x7f5254069db0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.260 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 --2- 172.21.15.112:0/3976235153 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52680b23f0 0x7f526813cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 >> 172.21.15.112:0/3976235153 conn(0x7f52680b5980 msgr2=0x7f526806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 shutdown_connections 2024-04-25T00:56:39.261 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.258+0000 7f526e9cf640 1 -- 172.21.15.112:0/3976235153 wait complete. 2024-04-25T00:56:39.276 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-04-25T00:56:37.530261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224096,"kb_used_data":3768,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750624,"statfs":{"total":773094113280,"available":772864638976,"internally_reserved":0,"allocated":3858432,"data_stored":2194614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2851,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-46,"num_write_kb":-15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001378"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":188,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-25T00:56:21.025163+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:56:21.025163+0000","last_peered":"2024-04-25T00:56:21.025163+0000","last_clean":"2024-04-25T00:56:21.025163+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:56:21.025163+0000","last_undegraded":"2024-04-25T00:56:21.025163+0000","last_fullsized":"2024-04-25T00:56:21.025163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":714470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":124116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":124082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":348160,"data_stored":134127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32072,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339768,"statfs":{"total":96636764160,"available":96603922432,"internally_reserved":0,"allocated":917504,"data_stored":714380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:56:39.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:56:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.386+0000 7fc4f4f2d640 1 Processor -- start 2024-04-25T00:56:39.391 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4f4f2d640 1 -- start start 2024-04-25T00:56:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4f4f2d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fc4f00b2100 0x7fc4f00b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4f4f2d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc4f014cbb0 0x7fc4f014efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4f4f2d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f00b2ac0 con 0x7fc4f00b47f0 2024-04-25T00:56:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4f4f2d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f00aff80 con 0x7fc4f014cbb0 2024-04-25T00:56:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4f4f2d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f00b1c70 con 0x7fc4f00b2100 2024-04-25T00:56:39.393 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4ee59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc4f00b47f0 0x7fc4f0147ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32820/0 (socket says 172.21.15.112:32820) 2024-04-25T00:56:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4ee59c640 1 -- 172.21.15.112:0/2809765701 learned_addr learned my addr 172.21.15.112:0/2809765701 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4eed9d640 1 --2- 172.21.15.112:0/2809765701 >> v2:172.21.15.112:3300/0 conn(0x7fc4f014cbb0 0x7fc4f014efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4edd9b640 1 --2- 172.21.15.112:0/2809765701 >> v2:172.21.15.187:3300/0 conn(0x7fc4f00b2100 0x7fc4f00b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4ed59a640 1 -- 172.21.15.112:0/2809765701 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2046689406 0 0) 0x7fc4e00014a0 con 0x7fc4f00b47f0 2024-04-25T00:56:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4ed59a640 1 -- 172.21.15.112:0/2809765701 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4dc005740 con 0x7fc4f00b47f0 2024-04-25T00:56:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4eed9d640 1 -- 172.21.15.112:0/2809765701 >> v2:172.21.15.187:3300/0 conn(0x7fc4f00b2100 msgr2=0x7fc4f00b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:39.394 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4eed9d640 1 --2- 172.21.15.112:0/2809765701 >> v2:172.21.15.187:3300/0 conn(0x7fc4f00b2100 0x7fc4f00b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4eed9d640 1 -- 172.21.15.112:0/2809765701 >> v1:172.21.15.112:6789/0 conn(0x7fc4f00b47f0 legacy=0x7fc4f0147ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.395 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4eed9d640 1 -- 172.21.15.112:0/2809765701 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00b4bd0 con 0x7fc4f014cbb0 2024-04-25T00:56:39.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4eed9d640 1 --2- 172.21.15.112:0/2809765701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f014cbb0 0x7fc4f014efa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00b0950 tx=0x7fc4e801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae5e02c882c4eda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:39.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4ed59a640 1 -- 172.21.15.112:0/2809765701 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80b0070 con 0x7fc4f014cbb0 2024-04-25T00:56:39.396 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4ed59a640 1 -- 172.21.15.112:0/2809765701 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e80ab600 con 0x7fc4f014cbb0 2024-04-25T00:56:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.390+0000 7fc4ed59a640 1 -- 172.21.15.112:0/2809765701 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e80ab9b0 con 0x7fc4f014cbb0 2024-04-25T00:56:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/2809765701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f014cbb0 msgr2=0x7fc4f014efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/2809765701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f014cbb0 0x7fc4f014efa0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00b0950 tx=0x7fc4e801c3f0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/2809765701 shutdown_connections 2024-04-25T00:56:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/2809765701 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f014cbb0 0x7fc4f014efa0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/2809765701 >> v2:172.21.15.187:3300/0 conn(0x7fc4f00b2100 0x7fc4f00b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.397 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/2809765701 >> 172.21.15.112:0/2809765701 conn(0x7fc4f00b5980 msgr2=0x7fc4f00b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:39.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/2809765701 shutdown_connections 2024-04-25T00:56:39.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/2809765701 wait complete. 2024-04-25T00:56:39.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 Processor -- start 2024-04-25T00:56:39.398 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- start start 2024-04-25T00:56:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f00b2100 0x7fc4f00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f00b47f0 0x7fc4f01dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f014cbb0 0x7fc4f01dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00ae280 con 0x7fc4f00b2100 2024-04-25T00:56:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00b1900 con 0x7fc4f00b47f0 2024-04-25T00:56:39.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f00ae400 con 0x7fc4f014cbb0 2024-04-25T00:56:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f00b2100 0x7fc4f00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f00b2100 0x7fc4f00b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60946/0 (socket says 172.21.15.112:60946) 2024-04-25T00:56:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 -- 172.21.15.112:0/3285311606 learned_addr learned my addr 172.21.15.112:0/3285311606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4edd9b640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f00b47f0 0x7fc4f01dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4eed9d640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f014cbb0 0x7fc4f01dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 -- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f014cbb0 msgr2=0x7fc4f01dc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f014cbb0 0x7fc4f01dc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 -- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f00b47f0 msgr2=0x7fc4f01dbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f00b47f0 0x7fc4f01dbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 -- 172.21.15.112:0/3285311606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f00b81d0 con 0x7fc4f00b2100 2024-04-25T00:56:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4edd9b640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f00b47f0 0x7fc4f01dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4ee59c640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f00b2100 0x7fc4f00b78b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000cb50 tx=0x7fc4e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4e6ffd640 1 -- 172.21.15.112:0/3285311606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0017070 con 0x7fc4f00b2100 2024-04-25T00:56:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4e6ffd640 1 -- 172.21.15.112:0/3285311606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e000ad40 con 0x7fc4f00b2100 2024-04-25T00:56:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4e6ffd640 1 -- 172.21.15.112:0/3285311606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0012640 con 0x7fc4f00b2100 2024-04-25T00:56:39.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f01dba40 con 0x7fc4f00b2100 2024-04-25T00:56:39.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f00b6a90 con 0x7fc4f00b2100 2024-04-25T00:56:39.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.394+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f004f580 con 0x7fc4f00b2100 2024-04-25T00:56:39.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.398+0000 7fc4e6ffd640 1 -- 172.21.15.112:0/3285311606 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc4e001a070 con 0x7fc4f00b2100 2024-04-25T00:56:39.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.398+0000 7fc4e6ffd640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc4d00678c0 0x7fc4d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:39.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.398+0000 7fc4edd9b640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc4d00678c0 0x7fc4d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:39.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.398+0000 7fc4e6ffd640 1 -- 172.21.15.112:0/3285311606 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fc4e0085830 con 0x7fc4f00b2100 2024-04-25T00:56:39.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.398+0000 7fc4edd9b640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc4d00678c0 0x7fc4d0069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01479f0 tx=0x7fc4d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:39.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.402+0000 7fc4e6ffd640 1 -- 172.21.15.112:0/3285311606 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4f00b2ac0 con 0x7fc4f00b2100 2024-04-25T00:56:39.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.558+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4f014d410 con 0x7fc4f00b2100 2024-04-25T00:56:39.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.562+0000 7fc4e6ffd640 1 -- 172.21.15.112:0/3285311606 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+13070 (secure 0 0 0) 0x7fc4e0056670 con 0x7fc4f00b2100 2024-04-25T00:56:39.564 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:39.565 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":32,"fsid":"86f864a3-be50-4513-98ca-86783428a0f8","created":"2024-04-25T00:55:14.875144+0000","modified":"2024-04-25T00:56:36.615528+0000","last_up_change":"2024-04-25T00:55:26.648984+0000","last_in_change":"2024-04-25T00:55:17.184543+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:55:27.587648+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"460ba393-4ad9-444f-bc8a-fafa413c291d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6806","nonce":1219565350}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6810","nonce":1219565350}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6815","nonce":1219565350}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1219565350},{"type":"v1","addr":"172.21.15.112:6813","nonce":1219565350}]},"public_addr":"172.21.15.112:6806/1219565350","cluster_addr":"172.21.15.112:6810/1219565350","heartbeat_back_addr":"172.21.15.112:6815/1219565350","heartbeat_front_addr":"172.21.15.112:6813/1219565350","state":["exists","up"]},{"osd":1,"uuid":"07888c5e-f032-433a-9294-06a2adec15bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6817","nonce":3975585121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6820","nonce":3975585121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6825","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6827","nonce":3975585121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":3975585121},{"type":"v1","addr":"172.21.15.112:6824","nonce":3975585121}]},"public_addr":"172.21.15.112:6817/3975585121","cluster_addr":"172.21.15.112:6820/3975585121","heartbeat_back_addr":"172.21.15.112:6827/3975585121","heartbeat_front_addr":"172.21.15.112:6824/3975585121","state":["exists","up"]},{"osd":2,"uuid":"73523f2d-5c2d-44dd-9227-05493e90372d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6819","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6821","nonce":3154807242}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6823","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6826","nonce":3154807242}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6831","nonce":3154807242}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":3154807242},{"type":"v1","addr":"172.21.15.112:6829","nonce":3154807242}]},"public_addr":"172.21.15.112:6821/3154807242","cluster_addr":"172.21.15.112:6826/3154807242","heartbeat_back_addr":"172.21.15.112:6831/3154807242","heartbeat_front_addr":"172.21.15.112:6829/3154807242","state":["exists","up"]},{"osd":3,"uuid":"bf987335-4191-4de3-ac02-490605f11e83","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6801","nonce":3692662359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6803","nonce":3692662359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6809","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6811","nonce":3692662359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6805","nonce":3692662359},{"type":"v1","addr":"172.21.15.112:6807","nonce":3692662359}]},"public_addr":"172.21.15.112:6801/3692662359","cluster_addr":"172.21.15.112:6803/3692662359","heartbeat_back_addr":"172.21.15.112:6811/3692662359","heartbeat_front_addr":"172.21.15.112:6807/3692662359","state":["exists","up"]},{"osd":4,"uuid":"56e4594f-d522-4bb1-985c-b308abc0e315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6803","nonce":1453523944}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6807","nonce":1453523944}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6814","nonce":1453523944}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":1453523944},{"type":"v1","addr":"172.21.15.187:6810","nonce":1453523944}]},"public_addr":"172.21.15.187:6803/1453523944","cluster_addr":"172.21.15.187:6807/1453523944","heartbeat_back_addr":"172.21.15.187:6814/1453523944","heartbeat_front_addr":"172.21.15.187:6810/1453523944","state":["exists","up"]},{"osd":5,"uuid":"58cf3cc8-e445-4def-b50a-f502aa6a24c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6817","nonce":2456695186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6818","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6819","nonce":2456695186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6822","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6823","nonce":2456695186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6820","nonce":2456695186},{"type":"v1","addr":"172.21.15.187:6821","nonce":2456695186}]},"public_addr":"172.21.15.187:6817/2456695186","cluster_addr":"172.21.15.187:6819/2456695186","heartbeat_back_addr":"172.21.15.187:6823/2456695186","heartbeat_front_addr":"172.21.15.187:6821/2456695186","state":["exists","up"]},{"osd":6,"uuid":"35497ef1-a007-4874-a742-e7038034e1ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6802","nonce":718817690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6806","nonce":718817690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6815","nonce":718817690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":718817690},{"type":"v1","addr":"172.21.15.187:6811","nonce":718817690}]},"public_addr":"172.21.15.187:6802/718817690","cluster_addr":"172.21.15.187:6806/718817690","heartbeat_back_addr":"172.21.15.187:6815/718817690","heartbeat_front_addr":"172.21.15.187:6811/718817690","state":["exists","up"]},{"osd":7,"uuid":"507622e0-4d62-4865-95a0-1c25f5c3a209","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6824","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6825","nonce":2342593789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6826","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6827","nonce":2342593789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6830","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6831","nonce":2342593789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6828","nonce":2342593789},{"type":"v1","addr":"172.21.15.187:6829","nonce":2342593789}]},"public_addr":"172.21.15.187:6825/2342593789","cluster_addr":"172.21.15.187:6827/2342593789","heartbeat_back_addr":"172.21.15.187:6831/2342593789","heartbeat_front_addr":"172.21.15.187:6829/2342593789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.876079+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.845712+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.801463+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.702904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.827855+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.686962+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.770496+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:55:22.757617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.112:6835/3222796217":"2024-04-26T00:56:33.261111+0000","172.21.15.112:6834/3222796217":"2024-04-26T00:56:33.261111+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:56:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.562+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc4d00678c0 msgr2=0x7fc4d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.562+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc4d00678c0 0x7fc4d0069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc4f01479f0 tx=0x7fc4d8008040 comp rx=0 tx=0).stop 2024-04-25T00:56:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.562+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f00b2100 msgr2=0x7fc4f00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.562+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f00b2100 0x7fc4f00b78b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000cb50 tx=0x7fc4e000a430 comp rx=0 tx=0).stop 2024-04-25T00:56:39.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.562+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 shutdown_connections 2024-04-25T00:56:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.562+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc4f014cbb0 0x7fc4f01dc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.566+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc4d00678c0 0x7fc4d0069d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.566+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc4f00b47f0 0x7fc4f01dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.566+0000 7fc4f4f2d640 1 --2- 172.21.15.112:0/3285311606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc4f00b2100 0x7fc4f00b78b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.566+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 >> 172.21.15.112:0/3285311606 conn(0x7fc4f00b5980 msgr2=0x7fc4f0069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.566+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 shutdown_connections 2024-04-25T00:56:39.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:39.566+0000 7fc4f4f2d640 1 -- 172.21.15.112:0/3285311606 wait complete. 2024-04-25T00:56:40.584 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T00:56:40.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:56:40.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f255db98640 1 Processor -- start 2024-04-25T00:56:40.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f255db98640 1 -- start start 2024-04-25T00:56:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f255db98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813efd0 0x7f255813f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f255db98640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25580729b0 con 0x7f25580725d0 2024-04-25T00:56:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f255db98640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25580b0210 con 0x7f255814db00 2024-04-25T00:56:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f255db98640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25580b0980 con 0x7f255813efd0 2024-04-25T00:56:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f25577fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f25580725d0 0x7f2558148790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32836/0 (socket says 172.21.15.112:32836) 2024-04-25T00:56:40.720 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f25577fe640 1 -- 172.21.15.112:0/908535576 learned_addr learned my addr 172.21.15.112:0/908535576 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 714575655 0 0) 0x7f25580729b0 con 0x7f25580725d0 2024-04-25T00:56:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2540003530 con 0x7f25580725d0 2024-04-25T00:56:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.714+0000 7f2556ffd640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813efd0 0x7f255813f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2546516646 0 0) 0x7f2540003530 con 0x7f25580725d0 2024-04-25T00:56:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813efd0 msgr2=0x7f255813f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813efd0 0x7f255813f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 >> v1:172.21.15.187:6789/0 conn(0x7f255814db00 legacy=0x7f255814fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.721 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25580b0bb0 con 0x7f25580725d0 2024-04-25T00:56:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f2548001680 con 0x7f25580725d0 2024-04-25T00:56:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 >> v1:172.21.15.112:6789/0 conn(0x7f25580725d0 legacy=0x7f2558148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813efd0 0x7f25400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2540004020 0x7f2540004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.722 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540004da0 0x7f2540005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f254000bc80 con 0x7f2540004da0 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f254000be30 con 0x7f2540004020 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f254000c030 con 0x7f255813efd0 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2557fff640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540004da0 0x7f2540005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2556ffd640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2540004020 0x7f2540004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25577fe640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813efd0 0x7f25400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2557fff640 1 -- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813efd0 msgr2=0x7f25400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2557fff640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813efd0 0x7f25400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.723 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2557fff640 1 -- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2540004020 msgr2=0x7f2540004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:40.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2557fff640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f2540004020 0x7f2540004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.724 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2557fff640 1 -- 172.21.15.112:0/908535576 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2540003530 con 0x7f2540004da0 2024-04-25T00:56:40.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2557fff640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540004da0 0x7f2540005220 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f255806e8b0 tx=0x7f255001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd13fe62a2f7a4e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:40.727 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500b0070 con 0x7f2540004da0 2024-04-25T00:56:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2550003b80 con 0x7f2540004da0 2024-04-25T00:56:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f25567fc640 1 -- 172.21.15.112:0/908535576 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550001a90 con 0x7f2540004da0 2024-04-25T00:56:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f255db98640 1 -- 172.21.15.112:0/908535576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540004da0 msgr2=0x7f2540005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f255db98640 1 --2- 172.21.15.112:0/908535576 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2540004da0 0x7f2540005220 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f255806e8b0 tx=0x7f255001c700 comp rx=0 tx=0).stop 2024-04-25T00:56:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f2557fff640 1 -- 172.21.15.112:0/908535576 reap_dead start 2024-04-25T00:56:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f255db98640 1 -- 172.21.15.112:0/908535576 shutdown_connections 2024-04-25T00:56:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f255db98640 1 -- 172.21.15.112:0/908535576 >> 172.21.15.112:0/908535576 conn(0x7f25580baae0 msgr2=0x7f25580baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:40.728 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f255db98640 1 -- 172.21.15.112:0/908535576 shutdown_connections 2024-04-25T00:56:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.718+0000 7f255db98640 1 -- 172.21.15.112:0/908535576 wait complete. 2024-04-25T00:56:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 Processor -- start 2024-04-25T00:56:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 -- start start 2024-04-25T00:56:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f255814a640 0x7f25580ba2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813fbe0 0x7f2558139090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25581395d0 0x7f25581e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f25580b17f0 con 0x7f25581395d0 2024-04-25T00:56:40.729 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f25580b1670 con 0x7f255814a640 2024-04-25T00:56:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f25580b1a70 con 0x7f255813fbe0 2024-04-25T00:56:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25581395d0 0x7f25581e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25581395d0 0x7f25581e13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60958/0 (socket says 172.21.15.112:60958) 2024-04-25T00:56:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 -- 172.21.15.112:0/1826777245 learned_addr learned my addr 172.21.15.112:0/1826777245 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2556ffd640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813fbe0 0x7f2558139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f25577fe640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f255814a640 0x7f25580ba2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 -- 172.21.15.112:0/1826777245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813fbe0 msgr2=0x7f2558139090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.730 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f255813fbe0 0x7f2558139090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 -- 172.21.15.112:0/1826777245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f255814a640 msgr2=0x7f25580ba2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f255814a640 0x7f25580ba2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 -- 172.21.15.112:0/1826777245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2558139a60 con 0x7f25581395d0 2024-04-25T00:56:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f2557fff640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25581395d0 0x7f25581e13d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2550097ef0 tx=0x7f2540005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500bb070 con 0x7f25581395d0 2024-04-25T00:56:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2550003d30 con 0x7f25581395d0 2024-04-25T00:56:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2558142360 con 0x7f25581395d0 2024-04-25T00:56:40.731 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500b0040 con 0x7f25581395d0 2024-04-25T00:56:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2558142920 con 0x7f25581395d0 2024-04-25T00:56:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.722+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25581419a0 con 0x7f25581395d0 2024-04-25T00:56:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f25500be070 con 0x7f25581395d0 2024-04-25T00:56:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f253ffff640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25440678c0 0x7f2544069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f255011e910 con 0x7f25581395d0 2024-04-25T00:56:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f25577fe640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25440678c0 0x7f2544069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.732 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f253ffff640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f254406d780 0x7f254406fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 --> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f255000bf30 con 0x7f254406d780 2024-04-25T00:56:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f255011eda0 con 0x7f25581395d0 2024-04-25T00:56:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f2556ffd640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f254406d780 0x7f254406fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f25577fe640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25440678c0 0x7f2544069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2548000c10 tx=0x7f2548009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f2556ffd640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f254406d780 0x7f254406fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:40.733 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.726+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 <== osd.0 v2:172.21.15.112:6804/1219565350 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f255000bf30 con 0x7f254406d780 2024-04-25T00:56:40.741 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 --> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2558068970 con 0x7f254406d780 2024-04-25T00:56:40.742 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f253ffff640 1 -- 172.21.15.112:0/1826777245 <== osd.0 v2:172.21.15.112:6804/1219565350 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2558068970 con 0x7f254406d780 2024-04-25T00:56:40.742 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:40.742 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:56:40.742 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f254406d780 msgr2=0x7f254406fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.743 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6804/1219565350,v1:172.21.15.112:6806/1219565350] conn(0x7f254406d780 0x7f254406fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25440678c0 msgr2=0x7f2544069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f25440678c0 0x7f2544069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2548000c10 tx=0x7f2548009040 comp rx=0 tx=0).stop 2024-04-25T00:56:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25581395d0 msgr2=0x7f25581e13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 --2- 172.21.15.112:0/1826777245 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25581395d0 0x7f25581e13d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2550097ef0 tx=0x7f2540005840 comp rx=0 tx=0).stop 2024-04-25T00:56:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f2557fff640 1 -- 172.21.15.112:0/1826777245 reap_dead start 2024-04-25T00:56:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 shutdown_connections 2024-04-25T00:56:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 >> 172.21.15.112:0/1826777245 conn(0x7f25580baae0 msgr2=0x7f255813c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:40.744 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 shutdown_connections 2024-04-25T00:56:40.745 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.738+0000 7f255db98640 1 -- 172.21.15.112:0/1826777245 wait complete. 2024-04-25T00:56:40.758 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T00:56:40.869 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5ce5c2640 1 Processor -- start 2024-04-25T00:56:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5ce5c2640 1 -- start start 2024-04-25T00:56:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5ce5c2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5c80add70 0x7fd5c8145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5ce5c2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c80aeec0 con 0x7fd5c80b3ce0 2024-04-25T00:56:40.870 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5ce5c2640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c80af780 con 0x7fd5c80b23f0 2024-04-25T00:56:40.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5ce5c2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c80b27d0 con 0x7fd5c80add70 2024-04-25T00:56:40.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5c77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5c80add70 0x7fd5c8145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5c77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5c80add70 0x7fd5c8145200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60964/0 (socket says 172.21.15.112:60964) 2024-04-25T00:56:40.871 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5c77fe640 1 -- 172.21.15.112:0/4290908002 learned_addr learned my addr 172.21.15.112:0/4290908002 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:40.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5c77fe640 1 -- 172.21.15.112:0/4290908002 >> v1:172.21.15.187:6789/0 conn(0x7fd5c80b23f0 legacy=0x7fd5c8072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5c6ffd640 1 -- 172.21.15.112:0/4290908002 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2452482386 0 0) 0x7fd5c80aeec0 con 0x7fd5c80b3ce0 2024-04-25T00:56:40.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5c77fe640 1 -- 172.21.15.112:0/4290908002 >> v1:172.21.15.112:6789/0 conn(0x7fd5c80b3ce0 legacy=0x7fd5c814fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.872 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.866+0000 7fd5c77fe640 1 -- 172.21.15.112:0/4290908002 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80afa20 con 0x7fd5c80add70 2024-04-25T00:56:40.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 --2- 172.21.15.112:0/4290908002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80add70 0x7fd5c8145200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80af950 tx=0x7fd5b801cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac83400207b444b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:40.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c6ffd640 1 -- 172.21.15.112:0/4290908002 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80a9620 con 0x7fd5c80add70 2024-04-25T00:56:40.878 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c6ffd640 1 -- 172.21.15.112:0/4290908002 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b80a97c0 con 0x7fd5c80add70 2024-04-25T00:56:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c6ffd640 1 -- 172.21.15.112:0/4290908002 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80a9b90 con 0x7fd5c80add70 2024-04-25T00:56:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/4290908002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80add70 msgr2=0x7fd5c8145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 --2- 172.21.15.112:0/4290908002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80add70 0x7fd5c8145200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80af950 tx=0x7fd5b801cfa0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/4290908002 shutdown_connections 2024-04-25T00:56:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 --2- 172.21.15.112:0/4290908002 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80add70 0x7fd5c8145200 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/4290908002 >> 172.21.15.112:0/4290908002 conn(0x7fd5c80b5980 msgr2=0x7fd5c80b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/4290908002 shutdown_connections 2024-04-25T00:56:40.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/4290908002 wait complete. 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 Processor -- start 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- start start 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5c80add70 0x7fd5c809b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80b23f0 0x7fd5c81ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c80b3ce0 0x7fd5c81de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80b52d0 con 0x7fd5c80b23f0 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80b5150 con 0x7fd5c80b3ce0 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c8153a80 con 0x7fd5c80add70 2024-04-25T00:56:40.880 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80b23f0 0x7fd5c81ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80b23f0 0x7fd5c81ddc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60966/0 (socket says 172.21.15.112:60966) 2024-04-25T00:56:40.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 -- 172.21.15.112:0/2825258921 learned_addr learned my addr 172.21.15.112:0/2825258921 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:40.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 -- 172.21.15.112:0/2825258921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5c80add70 msgr2=0x7fd5c809b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5c80add70 0x7fd5c809b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 -- 172.21.15.112:0/2825258921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c80b3ce0 msgr2=0x7fd5c81de1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:40.881 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ccb5f640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c80b3ce0 0x7fd5c81de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c80b3ce0 0x7fd5c81de1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:40.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 -- 172.21.15.112:0/2825258921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c809c720 con 0x7fd5c80b23f0 2024-04-25T00:56:40.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ccb5f640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c80b3ce0 0x7fd5c81de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:40.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c77fe640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80b23f0 0x7fd5c81ddc70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80ab040 tx=0x7fd5b801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:40.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c4ff9640 1 -- 172.21.15.112:0/2825258921 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80bc650 con 0x7fd5c80b23f0 2024-04-25T00:56:40.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c4ff9640 1 -- 172.21.15.112:0/2825258921 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b80bc810 con 0x7fd5c80b23f0 2024-04-25T00:56:40.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c809aa30 con 0x7fd5c80b23f0 2024-04-25T00:56:40.882 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5c4ff9640 1 -- 172.21.15.112:0/2825258921 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b80bcc00 con 0x7fd5c80b23f0 2024-04-25T00:56:40.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.870+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c8142ad0 con 0x7fd5c80b23f0 2024-04-25T00:56:40.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.874+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c804f580 con 0x7fd5c80b23f0 2024-04-25T00:56:40.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.874+0000 7fd5c4ff9640 1 -- 172.21.15.112:0/2825258921 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd5b80c53f0 con 0x7fd5c80b23f0 2024-04-25T00:56:40.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.874+0000 7fd5c4ff9640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5b40678c0 0x7fd5b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:40.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.874+0000 7fd5c4ff9640 1 -- 172.21.15.112:0/2825258921 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fd5b81205a0 con 0x7fd5c80b23f0 2024-04-25T00:56:40.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.874+0000 7fd5c7fff640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5b40678c0 0x7fd5b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:40.883 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.874+0000 7fd5c7fff640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5b40678c0 0x7fd5b4069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc000f70 tx=0x7fd5bc0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:40.884 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:40.878+0000 7fd5c4ff9640 1 -- 172.21.15.112:0/2825258921 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd5b80cb080 con 0x7fd5c80b23f0 2024-04-25T00:56:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.022+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fd5c8150750 con 0x7fd5b40678c0 2024-04-25T00:56:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.022+0000 7fd5c4ff9640 1 -- 172.21.15.112:0/2825258921 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd5c8150750 con 0x7fd5b40678c0 2024-04-25T00:56:41.027 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T00:56:41.028 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5b40678c0 msgr2=0x7fd5b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5b40678c0 0x7fd5b4069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc000f70 tx=0x7fd5bc0079c0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80b23f0 msgr2=0x7fd5c81ddc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80b23f0 0x7fd5c81ddc70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80ab040 tx=0x7fd5b801ad80 comp rx=0 tx=0).stop 2024-04-25T00:56:41.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 shutdown_connections 2024-04-25T00:56:41.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fd5b40678c0 0x7fd5b4069d80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fd5c80b3ce0 0x7fd5c81de1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5c80b23f0 0x7fd5c81ddc70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.029 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 --2- 172.21.15.112:0/2825258921 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fd5c80add70 0x7fd5c809b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 >> 172.21.15.112:0/2825258921 conn(0x7fd5c80b5980 msgr2=0x7fd5c8148f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 shutdown_connections 2024-04-25T00:56:41.030 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.026+0000 7fd5ce5c2640 1 -- 172.21.15.112:0/2825258921 wait complete. 2024-04-25T00:56:41.044 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T00:56:41.044 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:56:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde1d39d640 1 Processor -- start 2024-04-25T00:56:41.167 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde1d39d640 1 -- start start 2024-04-25T00:56:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde1d39d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fde181475c0 0x7fde180ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde1d39d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 0x7fde18144bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde1d39d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde18074a90 con 0x7fde180746b0 2024-04-25T00:56:41.170 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde1d39d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fde180ae7b0 con 0x7fde181475c0 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde1d39d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde180afc00 con 0x7fde18144760 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde16ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fde180746b0 0x7fde180b54e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32846/0 (socket says 172.21.15.112:32846) 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde16ffd640 1 -- 172.21.15.112:0/1058528882 learned_addr learned my addr 172.21.15.112:0/1058528882 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde167fc640 1 --2- 172.21.15.112:0/1058528882 >> v2:172.21.15.187:3300/0 conn(0x7fde181475c0 0x7fde180ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3699691807 0 0) 0x7fde18074a90 con 0x7fde180746b0 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde177fe640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 0x7fde18144bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde00003530 con 0x7fde180746b0 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1490452756 0 0) 0x7fde00003530 con 0x7fde180746b0 2024-04-25T00:56:41.171 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 msgr2=0x7fde18144bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 0x7fde18144bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 >> v2:172.21.15.187:3300/0 conn(0x7fde181475c0 msgr2=0x7fde180ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 --2- 172.21.15.112:0/1058528882 >> v2:172.21.15.187:3300/0 conn(0x7fde181475c0 0x7fde180ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde180aea00 con 0x7fde180746b0 2024-04-25T00:56:41.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7fde08001680 con 0x7fde180746b0 2024-04-25T00:56:41.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 >> v1:172.21.15.112:6789/0 conn(0x7fde180746b0 legacy=0x7fde180b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde000039c0 0x7fde00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.172 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 0x7fde00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde00004d50 0x7fde000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde180ae7b0 con 0x7fde000039c0 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde180afc00 con 0x7fde00004d50 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde0000ba50 con 0x7fde18144760 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde16ffd640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde000039c0 0x7fde00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde177fe640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde00004d50 0x7fde000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde167fc640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 0x7fde00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde16ffd640 1 -- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 msgr2=0x7fde00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.173 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde16ffd640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 0x7fde00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde16ffd640 1 -- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde00004d50 msgr2=0x7fde000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde16ffd640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde00004d50 0x7fde000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde16ffd640 1 -- 172.21.15.112:0/1058528882 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde00003530 con 0x7fde000039c0 2024-04-25T00:56:41.174 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.166+0000 7fde167fc640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde18144760 0x7fde00004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:41.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde16ffd640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde000039c0 0x7fde00003de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fde08001060 tx=0x7fde0801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c73bbd55292a966c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08007d80 con 0x7fde000039c0 2024-04-25T00:56:41.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde080a9020 con 0x7fde000039c0 2024-04-25T00:56:41.178 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde15ffb640 1 -- 172.21.15.112:0/1058528882 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08005b00 con 0x7fde000039c0 2024-04-25T00:56:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- 172.21.15.112:0/1058528882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde000039c0 msgr2=0x7fde00003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 --2- 172.21.15.112:0/1058528882 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde000039c0 0x7fde00003de0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fde08001060 tx=0x7fde0801cc70 comp rx=0 tx=0).stop 2024-04-25T00:56:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 -- 172.21.15.112:0/1058528882 reap_dead start 2024-04-25T00:56:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- 172.21.15.112:0/1058528882 shutdown_connections 2024-04-25T00:56:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- 172.21.15.112:0/1058528882 >> 172.21.15.112:0/1058528882 conn(0x7fde180b9240 msgr2=0x7fde180b9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- 172.21.15.112:0/1058528882 shutdown_connections 2024-04-25T00:56:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- 172.21.15.112:0/1058528882 wait complete. 2024-04-25T00:56:41.179 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 Processor -- start 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- start start 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde000039c0 0x7fde1813fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde180b1fd0 0x7fde181df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde181dfee0 0x7fde1813c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fde18152670 con 0x7fde181dfee0 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fde180af880 con 0x7fde180b1fd0 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde1d39d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fde181527f0 con 0x7fde000039c0 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde181dfee0 0x7fde1813c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.180 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde181dfee0 0x7fde1813c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60986/0 (socket says 172.21.15.112:60986) 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 -- 172.21.15.112:0/2916033251 learned_addr learned my addr 172.21.15.112:0/2916033251 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde16ffd640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde000039c0 0x7fde1813fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde167fc640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde180b1fd0 0x7fde181df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 -- 172.21.15.112:0/2916033251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde000039c0 msgr2=0x7fde1813fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fde000039c0 0x7fde1813fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 -- 172.21.15.112:0/2916033251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde180b1fd0 msgr2=0x7fde181df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fde180b1fd0 0x7fde181df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.170+0000 7fde177fe640 1 -- 172.21.15.112:0/2916033251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde180b37c0 con 0x7fde181dfee0 2024-04-25T00:56:41.181 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fde177fe640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde181dfee0 0x7fde1813c3b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fde10007c80 tx=0x7fde100115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde10011de0 con 0x7fde181dfee0 2024-04-25T00:56:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde1001b070 con 0x7fde181dfee0 2024-04-25T00:56:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde180b24e0 con 0x7fde181dfee0 2024-04-25T00:56:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde100039a0 con 0x7fde181dfee0 2024-04-25T00:56:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde180b2c50 con 0x7fde181dfee0 2024-04-25T00:56:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fde18068970 con 0x7fde181dfee0 2024-04-25T00:56:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fde10003b40 con 0x7fde181dfee0 2024-04-25T00:56:41.182 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fde040678c0 0x7fde04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fde10085f70 con 0x7fde181dfee0 2024-04-25T00:56:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fde16ffd640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fde040678c0 0x7fde04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7fde0406d780 0x7fde0406fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 --> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fde10011020 con 0x7fde0406d780 2024-04-25T00:56:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde10088610 con 0x7fde181dfee0 2024-04-25T00:56:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fde167fc640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7fde0406d780 0x7fde0406fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fde16ffd640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fde040678c0 0x7fde04069d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fde080aa000 tx=0x7fde08001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.183 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fde167fc640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7fde0406d780 0x7fde0406fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.184 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.174+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 <== osd.1 v2:172.21.15.112:6816/3975585121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fde10011020 con 0x7fde0406d780 2024-04-25T00:56:41.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.186+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 --> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fde18145940 con 0x7fde0406d780 2024-04-25T00:56:41.190 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fddff7fe640 1 -- 172.21.15.112:0/2916033251 <== osd.1 v2:172.21.15.112:6816/3975585121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fde18145940 con 0x7fde0406d780 2024-04-25T00:56:41.191 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:41.191 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:56:41.191 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:41.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7fde0406d780 msgr2=0x7fde0406fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6816/3975585121,v1:172.21.15.112:6817/3975585121] conn(0x7fde0406d780 0x7fde0406fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fde040678c0 msgr2=0x7fde04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fde040678c0 0x7fde04069d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fde080aa000 tx=0x7fde08001a50 comp rx=0 tx=0).stop 2024-04-25T00:56:41.192 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde181dfee0 msgr2=0x7fde1813c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 --2- 172.21.15.112:0/2916033251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde181dfee0 0x7fde1813c3b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fde10007c80 tx=0x7fde100115a0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde177fe640 1 -- 172.21.15.112:0/2916033251 reap_dead start 2024-04-25T00:56:41.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 shutdown_connections 2024-04-25T00:56:41.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 >> 172.21.15.112:0/2916033251 conn(0x7fde180b9240 msgr2=0x7fde180b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 shutdown_connections 2024-04-25T00:56:41.193 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.190+0000 7fde1d39d640 1 -- 172.21.15.112:0/2916033251 wait complete. 2024-04-25T00:56:41.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T00:56:41.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52ce4dc640 1 Processor -- start 2024-04-25T00:56:41.315 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52ce4dc640 1 -- start start 2024-04-25T00:56:41.316 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52ce4dc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f52c806c5e0 0x7f52c80af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52ce4dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 0x7f52c8072760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52ce4dc640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52c806c9c0 con 0x7f52c8072ca0 2024-04-25T00:56:41.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52ce4dc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f52c80b5110 con 0x7f52c806c5e0 2024-04-25T00:56:41.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52ce4dc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52c80afc60 con 0x7f52c8148700 2024-04-25T00:56:41.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f52c806c5e0 0x7f52c80af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c7fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f52c806c5e0 0x7f52c80af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60992/0 (socket says 172.21.15.112:60992) 2024-04-25T00:56:41.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c7fff640 1 -- 172.21.15.112:0/1706509987 learned_addr learned my addr 172.21.15.112:0/1706509987 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:41.317 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c7fff640 1 -- 172.21.15.112:0/1706509987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 msgr2=0x7f52c8072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c7fff640 1 --2- 172.21.15.112:0/1706509987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 0x7f52c8072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c7fff640 1 -- 172.21.15.112:0/1706509987 >> v1:172.21.15.187:6789/0 conn(0x7f52c8072ca0 legacy=0x7f52c8145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.318 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c7fff640 1 -- 172.21.15.112:0/1706509987 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c80aede0 con 0x7f52c806c5e0 2024-04-25T00:56:41.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c7fff640 1 --2- 172.21.15.112:0/1706509987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c806c5e0 0x7f52c80af720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009870 tx=0x7f52bc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd99219f6a7e9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.314+0000 7f52c6ffd640 1 -- 172.21.15.112:0/1706509987 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0b0070 con 0x7f52c806c5e0 2024-04-25T00:56:41.319 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c6ffd640 1 -- 172.21.15.112:0/1706509987 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52bc0ab600 con 0x7f52c806c5e0 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c6ffd640 1 -- 172.21.15.112:0/1706509987 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0ab9d0 con 0x7f52c806c5e0 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1706509987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c806c5e0 msgr2=0x7f52c80af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1706509987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c806c5e0 0x7f52c80af720 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009870 tx=0x7f52bc01c490 comp rx=0 tx=0).stop 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1706509987 shutdown_connections 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1706509987 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 0x7f52c8072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1706509987 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c806c5e0 0x7f52c80af720 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1706509987 >> 172.21.15.112:0/1706509987 conn(0x7f52c80b5980 msgr2=0x7f52c80b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1706509987 shutdown_connections 2024-04-25T00:56:41.320 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1706509987 wait complete. 2024-04-25T00:56:41.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 Processor -- start 2024-04-25T00:56:41.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- start start 2024-04-25T00:56:41.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52c806c5e0 0x7f52c809aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c8072ca0 0x7f52c8142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 0x7f52c809bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.323 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8152c30 con 0x7f52c8072ca0 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8152ab0 con 0x7f52c806c5e0 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52c8152db0 con 0x7f52c8148700 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c8072ca0 0x7f52c8142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c8072ca0 0x7f52c8142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60996/0 (socket says 172.21.15.112:60996) 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 -- 172.21.15.112:0/1574766466 learned_addr learned my addr 172.21.15.112:0/1574766466 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c7fff640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52c806c5e0 0x7f52c809aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 -- 172.21.15.112:0/1574766466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 msgr2=0x7f52c809bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52cca79640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 0x7f52c809bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.324 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 0x7f52c809bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 -- 172.21.15.112:0/1574766466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52c806c5e0 msgr2=0x7f52c809aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52c806c5e0 0x7f52c809aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 -- 172.21.15.112:0/1574766466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c809c520 con 0x7f52c8072ca0 2024-04-25T00:56:41.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c77fe640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c8072ca0 0x7f52c8142580 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f52b800b3f0 tx=0x7f52b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c4ff9640 1 -- 172.21.15.112:0/1574766466 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8015020 con 0x7f52c8072ca0 2024-04-25T00:56:41.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c4ff9640 1 -- 172.21.15.112:0/1574766466 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b8011830 con 0x7f52c8072ca0 2024-04-25T00:56:41.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c809c800 con 0x7f52c8072ca0 2024-04-25T00:56:41.325 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52c4ff9640 1 -- 172.21.15.112:0/1574766466 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8011c20 con 0x7f52c8072ca0 2024-04-25T00:56:41.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c813b800 con 0x7f52c8072ca0 2024-04-25T00:56:41.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.318+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c8142260 con 0x7f52c8072ca0 2024-04-25T00:56:41.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.322+0000 7f52c4ff9640 1 -- 172.21.15.112:0/1574766466 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f52b8018020 con 0x7f52c8072ca0 2024-04-25T00:56:41.326 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.322+0000 7f52c4ff9640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52b40678c0 0x7f52b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.322+0000 7f52c4ff9640 1 -- 172.21.15.112:0/1574766466 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f52b8085db0 con 0x7f52c8072ca0 2024-04-25T00:56:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.322+0000 7f52c7fff640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52b40678c0 0x7f52b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.322+0000 7f52c7fff640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52b40678c0 0x7f52b4069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0ad000 tx=0x7f52bc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.327 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.326+0000 7f52c4ff9640 1 -- 172.21.15.112:0/1574766466 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f52c8142260 con 0x7f52c8072ca0 2024-04-25T00:56:41.479 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f52c804f580 con 0x7f52b40678c0 2024-04-25T00:56:41.480 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52c4ff9640 1 -- 172.21.15.112:0/1574766466 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f52c804f580 con 0x7f52b40678c0 2024-04-25T00:56:41.480 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T00:56:41.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52b40678c0 msgr2=0x7f52b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52b40678c0 0x7f52b4069d80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f52bc0ad000 tx=0x7f52bc0a6000 comp rx=0 tx=0).stop 2024-04-25T00:56:41.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c8072ca0 msgr2=0x7f52c8142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.482 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c8072ca0 0x7f52c8142580 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f52b800b3f0 tx=0x7f52b800b8b0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 shutdown_connections 2024-04-25T00:56:41.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52c8148700 0x7f52c809bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52c8072ca0 0x7f52c8142580 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52b40678c0 0x7f52b4069d80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 --2- 172.21.15.112:0/1574766466 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52c806c5e0 0x7f52c809aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 >> 172.21.15.112:0/1574766466 conn(0x7f52c80b5980 msgr2=0x7f52c8147390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 shutdown_connections 2024-04-25T00:56:41.483 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.478+0000 7f52ce4dc640 1 -- 172.21.15.112:0/1574766466 wait complete. 2024-04-25T00:56:41.497 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T00:56:41.498 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:56:41.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f14783f9640 1 Processor -- start 2024-04-25T00:56:41.620 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f14783f9640 1 -- start start 2024-04-25T00:56:41.621 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f14783f9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f14700b21e0 0x7f1470148700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f14783f9640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f14700b4d90 0x7f147013ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f14783f9640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14700b25c0 con 0x7f147013f4a0 2024-04-25T00:56:41.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f14783f9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f147013f930 con 0x7f14700b21e0 2024-04-25T00:56:41.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f14783f9640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f147006f100 con 0x7f14700b4d90 2024-04-25T00:56:41.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f1476195640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f14700b21e0 0x7f1470148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.622 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f1476195640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f14700b21e0 0x7f1470148700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:60998/0 (socket says 172.21.15.112:60998) 2024-04-25T00:56:41.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f1476195640 1 -- 172.21.15.112:0/4004632768 learned_addr learned my addr 172.21.15.112:0/4004632768 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:41.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f1475994640 1 --2- 172.21.15.112:0/4004632768 >> v2:172.21.15.187:3300/0 conn(0x7f14700b4d90 0x7f147013ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f1476195640 1 -- 172.21.15.112:0/4004632768 >> v2:172.21.15.187:3300/0 conn(0x7f14700b4d90 msgr2=0x7f147013ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f1476195640 1 --2- 172.21.15.112:0/4004632768 >> v2:172.21.15.187:3300/0 conn(0x7f14700b4d90 0x7f147013ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f1476195640 1 -- 172.21.15.112:0/4004632768 >> v1:172.21.15.187:6789/0 conn(0x7f147013f4a0 legacy=0x7f1470145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.623 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.618+0000 7f1476195640 1 -- 172.21.15.112:0/4004632768 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147006f630 con 0x7f14700b21e0 2024-04-25T00:56:41.624 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476195640 1 --2- 172.21.15.112:0/4004632768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14700b21e0 0x7f1470148700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f147006fcc0 tx=0x7f146001c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4840d2998e82bd95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1475193640 1 -- 172.21.15.112:0/4004632768 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600a56c0 con 0x7f14700b21e0 2024-04-25T00:56:41.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1475193640 1 -- 172.21.15.112:0/4004632768 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14600a5860 con 0x7f14700b21e0 2024-04-25T00:56:41.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1475193640 1 -- 172.21.15.112:0/4004632768 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14600a5c30 con 0x7f14700b21e0 2024-04-25T00:56:41.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- 172.21.15.112:0/4004632768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14700b21e0 msgr2=0x7f1470148700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 --2- 172.21.15.112:0/4004632768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14700b21e0 0x7f1470148700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f147006fcc0 tx=0x7f146001c860 comp rx=0 tx=0).stop 2024-04-25T00:56:41.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- 172.21.15.112:0/4004632768 shutdown_connections 2024-04-25T00:56:41.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 --2- 172.21.15.112:0/4004632768 >> v2:172.21.15.187:3300/0 conn(0x7f14700b4d90 0x7f147013ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.625 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 --2- 172.21.15.112:0/4004632768 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14700b21e0 0x7f1470148700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- 172.21.15.112:0/4004632768 >> 172.21.15.112:0/4004632768 conn(0x7f14700b8930 msgr2=0x7f14700b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- 172.21.15.112:0/4004632768 shutdown_connections 2024-04-25T00:56:41.626 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- 172.21.15.112:0/4004632768 wait complete. 2024-04-25T00:56:41.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 Processor -- start 2024-04-25T00:56:41.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- start start 2024-04-25T00:56:41.628 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14700b21e0 0x7f1470144880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147013f4a0 0x7f1470072790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701e13e0 0x7f14701e1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f147006d790 con 0x7f14701e13e0 2024-04-25T00:56:41.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1470148c40 con 0x7f147013f4a0 2024-04-25T00:56:41.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f14783f9640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f14700b4990 con 0x7f14700b21e0 2024-04-25T00:56:41.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476996640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701e13e0 0x7f14701e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476996640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701e13e0 0x7f14701e1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32776/0 (socket says 172.21.15.112:32776) 2024-04-25T00:56:41.629 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476996640 1 -- 172.21.15.112:0/1916812071 learned_addr learned my addr 172.21.15.112:0/1916812071 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476195640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14700b21e0 0x7f1470144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1475994640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147013f4a0 0x7f1470072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476996640 1 -- 172.21.15.112:0/1916812071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14700b21e0 msgr2=0x7f1470144880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476996640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f14700b21e0 0x7f1470144880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476996640 1 -- 172.21.15.112:0/1916812071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147013f4a0 msgr2=0x7f1470072790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476996640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147013f4a0 0x7f1470072790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1476996640 1 -- 172.21.15.112:0/1916812071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1470144110 con 0x7f14701e13e0 2024-04-25T00:56:41.630 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.622+0000 7f1475994640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f147013f4a0 0x7f1470072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:41.631 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f1476996640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701e13e0 0x7f14701e1870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f146800b410 tx=0x7f146800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.633 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468015020 con 0x7f14701e13e0 2024-04-25T00:56:41.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1468011830 con 0x7f14701e13e0 2024-04-25T00:56:41.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1470144ec0 con 0x7f14701e13e0 2024-04-25T00:56:41.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468011c60 con 0x7f14701e13e0 2024-04-25T00:56:41.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1470143d50 con 0x7f14701e13e0 2024-04-25T00:56:41.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f14701e0fc0 con 0x7f14701e13e0 2024-04-25T00:56:41.634 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1468018020 con 0x7f14701e13e0 2024-04-25T00:56:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f145c0678c0 0x7f145c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f14680859d0 con 0x7f14701e13e0 2024-04-25T00:56:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f1476195640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f145c0678c0 0x7f145c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f145c06d780 0x7f145c06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 --> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f146801deb0 con 0x7f145c06d780 2024-04-25T00:56:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1468085e60 con 0x7f14701e13e0 2024-04-25T00:56:41.636 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f1475994640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f145c06d780 0x7f145c06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f1476195640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f145c0678c0 0x7f145c069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f14600ae000 tx=0x7f14600a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f1475994640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f145c06d780 0x7f145c06fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.637 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.626+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 <== osd.2 v2:172.21.15.112:6819/3154807242 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f146801deb0 con 0x7f145c06d780 2024-04-25T00:56:41.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 --> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1470068260 con 0x7f145c06d780 2024-04-25T00:56:41.644 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f146effd640 1 -- 172.21.15.112:0/1916812071 <== osd.2 v2:172.21.15.112:6819/3154807242 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1470068260 con 0x7f145c06d780 2024-04-25T00:56:41.644 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:41.645 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:56:41.645 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:41.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f145c06d780 msgr2=0x7f145c06fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6819/3154807242,v1:172.21.15.112:6821/3154807242] conn(0x7f145c06d780 0x7f145c06fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.646 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f145c0678c0 msgr2=0x7f145c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f145c0678c0 0x7f145c069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f14600ae000 tx=0x7f14600a7070 comp rx=0 tx=0).stop 2024-04-25T00:56:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701e13e0 msgr2=0x7f14701e1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 --2- 172.21.15.112:0/1916812071 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14701e13e0 0x7f14701e1870 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f146800b410 tx=0x7f146800b8d0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f1476996640 1 -- 172.21.15.112:0/1916812071 reap_dead start 2024-04-25T00:56:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 shutdown_connections 2024-04-25T00:56:41.647 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 >> 172.21.15.112:0/1916812071 conn(0x7f14700b8930 msgr2=0x7f1470069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 shutdown_connections 2024-04-25T00:56:41.648 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.642+0000 7f14783f9640 1 -- 172.21.15.112:0/1916812071 wait complete. 2024-04-25T00:56:41.659 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T00:56:41.761 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d9b44640 1 Processor -- start 2024-04-25T00:56:41.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d9b44640 1 -- start start 2024-04-25T00:56:41.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d9b44640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40add70 0x7f08d4145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d9b44640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08d40b3ce0 0x7f08d414fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d9b44640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d40aeec0 con 0x7f08d40b23f0 2024-04-25T00:56:41.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d9b44640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f08d40af780 con 0x7f08d40b3ce0 2024-04-25T00:56:41.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d9b44640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08d40b4170 con 0x7f08d40add70 2024-04-25T00:56:41.762 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08d40b3ce0 0x7f08d414fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d2ffd640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40add70 0x7f08d4145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d3fff640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f08d40b3ce0 0x7f08d414fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.187:3300/0 says I am v2:172.21.15.112:40214/0 (socket says 172.21.15.112:40214) 2024-04-25T00:56:41.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d37fe640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f08d40b23f0 0x7f08d4072760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:38878/0 (socket says 172.21.15.112:38878) 2024-04-25T00:56:41.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d3fff640 1 -- 172.21.15.112:0/466672972 learned_addr learned my addr 172.21.15.112:0/466672972 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:41.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d3fff640 1 -- 172.21.15.112:0/466672972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40add70 msgr2=0x7f08d4145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:41.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d3fff640 1 --2- 172.21.15.112:0/466672972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40add70 0x7f08d4145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d3fff640 1 -- 172.21.15.112:0/466672972 >> v1:172.21.15.187:6789/0 conn(0x7f08d40b23f0 legacy=0x7f08d4072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.763 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.758+0000 7f08d3fff640 1 -- 172.21.15.112:0/466672972 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d40b27d0 con 0x7f08d40b3ce0 2024-04-25T00:56:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d3fff640 1 --2- 172.21.15.112:0/466672972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b3ce0 0x7f08d414fb40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009e50 tx=0x7f08cc01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9629dd5dd4637cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d27fc640 1 -- 172.21.15.112:0/466672972 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0a5810 con 0x7f08d40b3ce0 2024-04-25T00:56:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d27fc640 1 -- 172.21.15.112:0/466672972 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08cc0a59b0 con 0x7f08d40b3ce0 2024-04-25T00:56:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d27fc640 1 -- 172.21.15.112:0/466672972 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0a5d80 con 0x7f08d40b3ce0 2024-04-25T00:56:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- 172.21.15.112:0/466672972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b3ce0 msgr2=0x7f08d414fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 --2- 172.21.15.112:0/466672972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b3ce0 0x7f08d414fb40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009e50 tx=0x7f08cc01ca30 comp rx=0 tx=0).stop 2024-04-25T00:56:41.765 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- 172.21.15.112:0/466672972 shutdown_connections 2024-04-25T00:56:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 --2- 172.21.15.112:0/466672972 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b3ce0 0x7f08d414fb40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 --2- 172.21.15.112:0/466672972 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40add70 0x7f08d4145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- 172.21.15.112:0/466672972 >> 172.21.15.112:0/466672972 conn(0x7f08d40b5980 msgr2=0x7f08d40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- 172.21.15.112:0/466672972 shutdown_connections 2024-04-25T00:56:41.766 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- 172.21.15.112:0/466672972 wait complete. 2024-04-25T00:56:41.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 Processor -- start 2024-04-25T00:56:41.768 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- start start 2024-04-25T00:56:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40add70 0x7f08d4147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b23f0 0x7f08d41398b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40b3ce0 0x7f08d41469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f08d40b52d0 con 0x7f08d40add70 2024-04-25T00:56:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f08d40b5150 con 0x7f08d40b23f0 2024-04-25T00:56:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.762+0000 7f08d9b44640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f08d41528a0 con 0x7f08d40b3ce0 2024-04-25T00:56:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40add70 0x7f08d4147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.769 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40add70 0x7f08d4147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32788/0 (socket says 172.21.15.112:32788) 2024-04-25T00:56:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 -- 172.21.15.112:0/3842490802 learned_addr learned my addr 172.21.15.112:0/3842490802 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d2ffd640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b23f0 0x7f08d41398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d3fff640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40b3ce0 0x7f08d41469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 -- 172.21.15.112:0/3842490802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40b3ce0 msgr2=0x7f08d41469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40b3ce0 0x7f08d41469d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 -- 172.21.15.112:0/3842490802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b23f0 msgr2=0x7f08d41398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b23f0 0x7f08d41398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.770 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 -- 172.21.15.112:0/3842490802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d4149170 con 0x7f08d40add70 2024-04-25T00:56:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d37fe640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40add70 0x7f08d4147250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c450 tx=0x7f08c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08bbfff640 1 -- 172.21.15.112:0/3842490802 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4018070 con 0x7f08d40add70 2024-04-25T00:56:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08bbfff640 1 -- 172.21.15.112:0/3842490802 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c4007480 con 0x7f08d40add70 2024-04-25T00:56:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08bbfff640 1 -- 172.21.15.112:0/3842490802 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c40047f0 con 0x7f08d40add70 2024-04-25T00:56:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d4147850 con 0x7f08d40add70 2024-04-25T00:56:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d4138380 con 0x7f08d40add70 2024-04-25T00:56:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d4147be0 con 0x7f08d40add70 2024-04-25T00:56:41.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08bbfff640 1 -- 172.21.15.112:0/3842490802 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f08c401b070 con 0x7f08d40add70 2024-04-25T00:56:41.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08bbfff640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f08c00678f0 0x7f08c0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:41.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08bbfff640 1 -- 172.21.15.112:0/3842490802 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f08c4084b10 con 0x7f08d40add70 2024-04-25T00:56:41.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d2ffd640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f08c00678f0 0x7f08c0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:41.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.766+0000 7f08d2ffd640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f08c00678f0 0x7f08c0069db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f08d4139620 tx=0x7f08c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:41.773 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.770+0000 7f08bbfff640 1 -- 172.21.15.112:0/3842490802 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f08c40559e0 con 0x7f08d40add70 2024-04-25T00:56:41.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.918+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f08d404f580 con 0x7f08c00678f0 2024-04-25T00:56:41.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.918+0000 7f08bbfff640 1 -- 172.21.15.112:0/3842490802 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f08d404f580 con 0x7f08c00678f0 2024-04-25T00:56:41.921 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T00:56:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f08c00678f0 msgr2=0x7f08c0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f08c00678f0 0x7f08c0069db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f08d4139620 tx=0x7f08c8008040 comp rx=0 tx=0).stop 2024-04-25T00:56:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40add70 msgr2=0x7f08d4147250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40add70 0x7f08d4147250 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f08c400c450 tx=0x7f08c400c910 comp rx=0 tx=0).stop 2024-04-25T00:56:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 shutdown_connections 2024-04-25T00:56:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f08d40b3ce0 0x7f08d41469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f08c00678f0 0x7f08c0069db0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f08d40b23f0 0x7f08d41398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 --2- 172.21.15.112:0/3842490802 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f08d40add70 0x7f08d4147250 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:41.924 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 >> 172.21.15.112:0/3842490802 conn(0x7f08d40b5980 msgr2=0x7f08d406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:41.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 shutdown_connections 2024-04-25T00:56:41.925 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:41.922+0000 7f08d9b44640 1 -- 172.21.15.112:0/3842490802 wait complete. 2024-04-25T00:56:41.938 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T00:56:41.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:56:42.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032f68d640 1 Processor -- start 2024-04-25T00:56:42.056 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032f68d640 1 -- start start 2024-04-25T00:56:42.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032f68d640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f03280b0b70 0x7f03281441a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032f68d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032809f180 0x7f03281446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032f68d640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032806d630 con 0x7f032806d250 2024-04-25T00:56:42.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032f68d640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f0328144c20 con 0x7f03280b0b70 2024-04-25T00:56:42.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032f68d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f03280af300 con 0x7f032809f180 2024-04-25T00:56:42.057 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032dc2a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032809f180 0x7f03281446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032cc28640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f03280b0b70 0x7f03281441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032d429640 1 --1- >> v1:172.21.15.187:6789/0 conn(0x7f032806d250 0x7f032809ea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.187:6789/0 says I am v1:172.21.15.112:38886/0 (socket says 172.21.15.112:38886) 2024-04-25T00:56:42.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032d429640 1 -- 172.21.15.112:0/845348235 learned_addr learned my addr 172.21.15.112:0/845348235 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:42.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032cc28640 1 -- 172.21.15.112:0/845348235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032809f180 msgr2=0x7f03281446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:42.058 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032cc28640 1 --2- 172.21.15.112:0/845348235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032809f180 0x7f03281446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032cc28640 1 -- 172.21.15.112:0/845348235 >> v1:172.21.15.187:6789/0 conn(0x7f032806d250 legacy=0x7f032809ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.054+0000 7f032cc28640 1 -- 172.21.15.112:0/845348235 --> v2:172.21.15.187:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03280b6250 con 0x7f03280b0b70 2024-04-25T00:56:42.059 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032cc28640 1 --2- 172.21.15.112:0/845348235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03280b0b70 0x7f03281441a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f031c009d90 tx=0x7f031c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac22731662dc85c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f0327fff640 1 -- 172.21.15.112:0/845348235 <== mon.1 v2:172.21.15.187:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c0b2070 con 0x7f03280b0b70 2024-04-25T00:56:42.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f0327fff640 1 -- 172.21.15.112:0/845348235 <== mon.1 v2:172.21.15.187:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f031c0a5900 con 0x7f03280b0b70 2024-04-25T00:56:42.060 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f0327fff640 1 -- 172.21.15.112:0/845348235 <== mon.1 v2:172.21.15.187:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c0a5cd0 con 0x7f03280b0b70 2024-04-25T00:56:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 -- 172.21.15.112:0/845348235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03280b0b70 msgr2=0x7f03281441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 --2- 172.21.15.112:0/845348235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03280b0b70 0x7f03281441a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f031c009d90 tx=0x7f031c01c970 comp rx=0 tx=0).stop 2024-04-25T00:56:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 -- 172.21.15.112:0/845348235 shutdown_connections 2024-04-25T00:56:42.061 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 --2- 172.21.15.112:0/845348235 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032809f180 0x7f03281446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 --2- 172.21.15.112:0/845348235 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f03280b0b70 0x7f03281441a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 -- 172.21.15.112:0/845348235 >> 172.21.15.112:0/845348235 conn(0x7f03280b6650 msgr2=0x7f03280b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 -- 172.21.15.112:0/845348235 shutdown_connections 2024-04-25T00:56:42.062 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 -- 172.21.15.112:0/845348235 wait complete. 2024-04-25T00:56:42.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.058+0000 7f032f68d640 1 Processor -- start 2024-04-25T00:56:42.063 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 -- start start 2024-04-25T00:56:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032806d250 0x7f03281e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f032809f180 0x7f032813b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032813cd10 0x7f03281ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0328072e30 con 0x7f032806d250 2024-04-25T00:56:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f0328072cb0 con 0x7f032809f180 2024-04-25T00:56:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0328151260 con 0x7f032813cd10 2024-04-25T00:56:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032806d250 0x7f03281e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.065 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032806d250 0x7f03281e1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32796/0 (socket says 172.21.15.112:32796) 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 -- 172.21.15.112:0/2448341456 learned_addr learned my addr 172.21.15.112:0/2448341456 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032dc2a640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032813cd10 0x7f03281ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032cc28640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f032809f180 0x7f032813b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 -- 172.21.15.112:0/2448341456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032813cd10 msgr2=0x7f03281ddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f032813cd10 0x7f03281ddbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 -- 172.21.15.112:0/2448341456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f032809f180 msgr2=0x7f032813b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f032809f180 0x7f032813b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 -- 172.21.15.112:0/2448341456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03281e11d0 con 0x7f032806d250 2024-04-25T00:56:42.066 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032d429640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032806d250 0x7f03281e1c40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f032000c450 tx=0x7f032000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320017070 con 0x7f032806d250 2024-04-25T00:56:42.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0320005cf0 con 0x7f032806d250 2024-04-25T00:56:42.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03281e1440 con 0x7f032806d250 2024-04-25T00:56:42.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320007290 con 0x7f032806d250 2024-04-25T00:56:42.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281389e0 con 0x7f032806d250 2024-04-25T00:56:42.067 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0328068970 con 0x7f032806d250 2024-04-25T00:56:42.068 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f032001a070 con 0x7f032806d250 2024-04-25T00:56:42.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03140678f0 0x7f0314069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f0320085340 con 0x7f032806d250 2024-04-25T00:56:42.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f032cc28640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03140678f0 0x7f0314069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f031406d7b0 0x7f031406fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 --> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0314070280 con 0x7f031406d7b0 2024-04-25T00:56:42.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.062+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f03200857d0 con 0x7f032806d250 2024-04-25T00:56:42.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.066+0000 7f032dc2a640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f031406d7b0 0x7f031406fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.069 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.066+0000 7f032cc28640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03140678f0 0x7f0314069db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f031c0af000 tx=0x7f031c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.066+0000 7f032dc2a640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f031406d7b0 0x7f031406fbd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.070 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.066+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 <== osd.3 v2:172.21.15.112:6800/3692662359 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0314070280 con 0x7f031406d7b0 2024-04-25T00:56:42.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 --> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f03280b5510 con 0x7f031406d7b0 2024-04-25T00:56:42.081 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f0325ffb640 1 -- 172.21.15.112:0/2448341456 <== osd.3 v2:172.21.15.112:6800/3692662359 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f03280b5510 con 0x7f031406d7b0 2024-04-25T00:56:42.081 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:42.081 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:56:42.081 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:42.082 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f031406d7b0 msgr2=0x7f031406fbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6800/3692662359,v1:172.21.15.112:6801/3692662359] conn(0x7f031406d7b0 0x7f031406fbd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03140678f0 msgr2=0x7f0314069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03140678f0 0x7f0314069db0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f031c0af000 tx=0x7f031c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:56:42.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032806d250 msgr2=0x7f03281e1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 --2- 172.21.15.112:0/2448341456 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f032806d250 0x7f03281e1c40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f032000c450 tx=0x7f032000c910 comp rx=0 tx=0).stop 2024-04-25T00:56:42.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032dc2a640 1 -- 172.21.15.112:0/2448341456 reap_dead start 2024-04-25T00:56:42.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 shutdown_connections 2024-04-25T00:56:42.083 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 >> 172.21.15.112:0/2448341456 conn(0x7f03280b6650 msgr2=0x7f032806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 shutdown_connections 2024-04-25T00:56:42.084 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.078+0000 7f032f68d640 1 -- 172.21.15.112:0/2448341456 wait complete. 2024-04-25T00:56:42.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T00:56:42.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d284ae640 1 Processor -- start 2024-04-25T00:56:42.211 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d284ae640 1 -- start start 2024-04-25T00:56:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d284ae640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f1d2009c270 0x7f1d20147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d284ae640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1d20147650 0x7f1d20148240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d284ae640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d200bbec0 con 0x7f1d200722d0 2024-04-25T00:56:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d284ae640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f1d200baaf0 con 0x7f1d20147650 2024-04-25T00:56:42.213 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d284ae640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f1d200bc5c0 con 0x7f1d2009c270 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d26a4b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1d20147650 0x7f1d20148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d2624a640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f1d200722d0 0x7f1d20072bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32854/0 (socket says 172.21.15.112:32854) 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d2624a640 1 -- 172.21.15.112:0/464642965 learned_addr learned my addr 172.21.15.112:0/464642965 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d25a49640 1 --2- 172.21.15.112:0/464642965 >> v2:172.21.15.187:3300/0 conn(0x7f1d2009c270 0x7f1d20147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d26a4b640 1 -- 172.21.15.112:0/464642965 >> v2:172.21.15.187:3300/0 conn(0x7f1d2009c270 msgr2=0x7f1d20147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d26a4b640 1 --2- 172.21.15.112:0/464642965 >> v2:172.21.15.187:3300/0 conn(0x7f1d2009c270 0x7f1d20147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d26a4b640 1 -- 172.21.15.112:0/464642965 >> v1:172.21.15.112:6789/0 conn(0x7f1d200722d0 legacy=0x7f1d20072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d26a4b640 1 -- 172.21.15.112:0/464642965 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d200bc8a0 con 0x7f1d20147650 2024-04-25T00:56:42.214 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.210+0000 7f1d25248640 1 -- 172.21.15.112:0/464642965 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3945847057 0 0) 0x7f1d200bbec0 con 0x7f1d200722d0 2024-04-25T00:56:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d26a4b640 1 --2- 172.21.15.112:0/464642965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20147650 0x7f1d20148240 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1d200ba460 tx=0x7f1d1801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb394d79114a211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25248640 1 -- 172.21.15.112:0/464642965 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d180b2070 con 0x7f1d20147650 2024-04-25T00:56:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25248640 1 -- 172.21.15.112:0/464642965 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d180a59c0 con 0x7f1d20147650 2024-04-25T00:56:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25248640 1 -- 172.21.15.112:0/464642965 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d180a5d90 con 0x7f1d20147650 2024-04-25T00:56:42.216 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- 172.21.15.112:0/464642965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20147650 msgr2=0x7f1d20148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 --2- 172.21.15.112:0/464642965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20147650 0x7f1d20148240 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f1d200ba460 tx=0x7f1d1801c900 comp rx=0 tx=0).stop 2024-04-25T00:56:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- 172.21.15.112:0/464642965 shutdown_connections 2024-04-25T00:56:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 --2- 172.21.15.112:0/464642965 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d20147650 0x7f1d20148240 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 --2- 172.21.15.112:0/464642965 >> v2:172.21.15.187:3300/0 conn(0x7f1d2009c270 0x7f1d20147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- 172.21.15.112:0/464642965 >> 172.21.15.112:0/464642965 conn(0x7f1d2009e8c0 msgr2=0x7f1d2009ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- 172.21.15.112:0/464642965 shutdown_connections 2024-04-25T00:56:42.217 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- 172.21.15.112:0/464642965 wait complete. 2024-04-25T00:56:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 Processor -- start 2024-04-25T00:56:42.220 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- start start 2024-04-25T00:56:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d200722d0 0x7f1d20143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2009c270 0x7f1d201e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d20147650 0x7f1d201e4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f1d20154890 con 0x7f1d2009c270 2024-04-25T00:56:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f1d20154710 con 0x7f1d200722d0 2024-04-25T00:56:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f1d20154a10 con 0x7f1d20147650 2024-04-25T00:56:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2009c270 0x7f1d201e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.221 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2009c270 0x7f1d201e3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32814/0 (socket says 172.21.15.112:32814) 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 -- 172.21.15.112:0/1114276066 learned_addr learned my addr 172.21.15.112:0/1114276066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d2624a640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d200722d0 0x7f1d20143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d26a4b640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d20147650 0x7f1d201e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 -- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d20147650 msgr2=0x7f1d201e4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d20147650 0x7f1d201e4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 -- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d200722d0 msgr2=0x7f1d20143fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d200722d0 0x7f1d20143fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 -- 172.21.15.112:0/1114276066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d200a2a50 con 0x7f1d2009c270 2024-04-25T00:56:42.222 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d2624a640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d200722d0 0x7f1d20143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:42.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d25a49640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2009c270 0x7f1d201e3df0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00d800 tx=0x7f1d1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d16ffd640 1 -- 172.21.15.112:0/1114276066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00b6c0 con 0x7f1d2009c270 2024-04-25T00:56:42.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d16ffd640 1 -- 172.21.15.112:0/1114276066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1c019070 con 0x7f1d2009c270 2024-04-25T00:56:42.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d200b6b90 con 0x7f1d2009c270 2024-04-25T00:56:42.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d16ffd640 1 -- 172.21.15.112:0/1114276066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c00ba60 con 0x7f1d2009c270 2024-04-25T00:56:42.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.214+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d201449c0 con 0x7f1d2009c270 2024-04-25T00:56:42.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.218+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d200b6990 con 0x7f1d2009c270 2024-04-25T00:56:42.223 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.218+0000 7f1d16ffd640 1 -- 172.21.15.112:0/1114276066 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1d1c00bc00 con 0x7f1d2009c270 2024-04-25T00:56:42.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.218+0000 7f1d16ffd640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1d040678f0 0x7f1d04069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.218+0000 7f1d2624a640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1d040678f0 0x7f1d04069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.218+0000 7f1d16ffd640 1 -- 172.21.15.112:0/1114276066 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f1d1c085da0 con 0x7f1d2009c270 2024-04-25T00:56:42.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.218+0000 7f1d2624a640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1d040678f0 0x7f1d04069db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1d10007fd0 tx=0x7f1d10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.224 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.222+0000 7f1d16ffd640 1 -- 172.21.15.112:0/1114276066 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1d1c056c70 con 0x7f1d2009c270 2024-04-25T00:56:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f1d2004f580 con 0x7f1d040678f0 2024-04-25T00:56:42.375 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d16ffd640 1 -- 172.21.15.112:0/1114276066 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1d2004f580 con 0x7f1d040678f0 2024-04-25T00:56:42.376 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T00:56:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1d040678f0 msgr2=0x7f1d04069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1d040678f0 0x7f1d04069db0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1d10007fd0 tx=0x7f1d10009040 comp rx=0 tx=0).stop 2024-04-25T00:56:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2009c270 msgr2=0x7f1d201e3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2009c270 0x7f1d201e3df0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00d800 tx=0x7f1d1c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 shutdown_connections 2024-04-25T00:56:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f1d20147650 0x7f1d201e4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.378 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d2009c270 0x7f1d201e3df0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f1d040678f0 0x7f1d04069db0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 --2- 172.21.15.112:0/1114276066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f1d200722d0 0x7f1d20143fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 >> 172.21.15.112:0/1114276066 conn(0x7f1d2009e8c0 msgr2=0x7f1d20141150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 shutdown_connections 2024-04-25T00:56:42.379 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.374+0000 7f1d284ae640 1 -- 172.21.15.112:0/1114276066 wait complete. 2024-04-25T00:56:42.393 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T00:56:42.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:56:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa524fe6640 1 Processor -- start 2024-04-25T00:56:42.509 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa524fe6640 1 -- start start 2024-04-25T00:56:42.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa524fe6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 0x7fa5200ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa524fe6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa5201443f0 0x7fa520144880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.511 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa524fe6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa520074a70 con 0x7fa520074690 2024-04-25T00:56:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa524fe6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fa5200b5880 con 0x7fa5201443f0 2024-04-25T00:56:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa524fe6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa5200afeb0 con 0x7fa5200ae950 2024-04-25T00:56:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa51e59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fa520074690 0x7fa5200ae240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:32860/0 (socket says 172.21.15.112:32860) 2024-04-25T00:56:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa51e59c640 1 -- 172.21.15.112:0/1862608331 learned_addr learned my addr 172.21.15.112:0/1862608331 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa51ed9d640 1 --2- 172.21.15.112:0/1862608331 >> v2:172.21.15.112:3300/0 conn(0x7fa5201443f0 0x7fa520144880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa51dd9b640 1 --2- 172.21.15.112:0/1862608331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 0x7fa5200ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa51d59a640 1 -- 172.21.15.112:0/1862608331 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480570994 0 0) 0x7fa520074a70 con 0x7fa520074690 2024-04-25T00:56:42.512 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa51d59a640 1 -- 172.21.15.112:0/1862608331 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa508003530 con 0x7fa520074690 2024-04-25T00:56:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.506+0000 7fa51ed9d640 1 -- 172.21.15.112:0/1862608331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 msgr2=0x7fa5200ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa51ed9d640 1 --2- 172.21.15.112:0/1862608331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 0x7fa5200ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa51ed9d640 1 -- 172.21.15.112:0/1862608331 >> v1:172.21.15.112:6789/0 conn(0x7fa520074690 legacy=0x7fa5200ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa51ed9d640 1 -- 172.21.15.112:0/1862608331 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5200b5230 con 0x7fa5201443f0 2024-04-25T00:56:42.513 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa51dd9b640 1 --2- 172.21.15.112:0/1862608331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 0x7fa5200ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:42.514 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa51ed9d640 1 --2- 172.21.15.112:0/1862608331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 0x7fa520144880 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa518009d90 tx=0x7fa51801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18968b4724ca340b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa51d59a640 1 -- 172.21.15.112:0/1862608331 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5180b2070 con 0x7fa5201443f0 2024-04-25T00:56:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa51d59a640 1 -- 172.21.15.112:0/1862608331 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5180a58b0 con 0x7fa5201443f0 2024-04-25T00:56:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa51d59a640 1 -- 172.21.15.112:0/1862608331 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5180a5c60 con 0x7fa5201443f0 2024-04-25T00:56:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa524fe6640 1 -- 172.21.15.112:0/1862608331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 msgr2=0x7fa520144880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa524fe6640 1 --2- 172.21.15.112:0/1862608331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 0x7fa520144880 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa518009d90 tx=0x7fa51801c8e0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa524fe6640 1 -- 172.21.15.112:0/1862608331 shutdown_connections 2024-04-25T00:56:42.515 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa524fe6640 1 --2- 172.21.15.112:0/1862608331 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 0x7fa520144880 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa524fe6640 1 --2- 172.21.15.112:0/1862608331 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 0x7fa5200ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa524fe6640 1 -- 172.21.15.112:0/1862608331 >> 172.21.15.112:0/1862608331 conn(0x7fa5200b5c30 msgr2=0x7fa5200b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa524fe6640 1 -- 172.21.15.112:0/1862608331 shutdown_connections 2024-04-25T00:56:42.516 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.510+0000 7fa524fe6640 1 -- 172.21.15.112:0/1862608331 wait complete. 2024-04-25T00:56:42.519 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 Processor -- start 2024-04-25T00:56:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 -- start start 2024-04-25T00:56:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa520074690 0x7fa5200b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 0x7fa5201e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 0x7fa5201e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fa5201546e0 con 0x7fa5201443f0 2024-04-25T00:56:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fa520154560 con 0x7fa520074690 2024-04-25T00:56:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fa520154860 con 0x7fa5200ae950 2024-04-25T00:56:42.520 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 0x7fa5201e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 0x7fa5201e31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32846/0 (socket says 172.21.15.112:32846) 2024-04-25T00:56:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 -- 172.21.15.112:0/1148423066 learned_addr learned my addr 172.21.15.112:0/1148423066 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51e59c640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa520074690 0x7fa5200b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51dd9b640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 0x7fa5201e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 -- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 msgr2=0x7fa5201e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fa5200ae950 0x7fa5201e2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 -- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa520074690 msgr2=0x7fa5200b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.521 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fa520074690 0x7fa5200b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 -- 172.21.15.112:0/1148423066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5200b1650 con 0x7fa5201443f0 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51ed9d640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 0x7fa5201e31b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa5180b0040 tx=0x7fa518019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5180b2050 con 0x7fa5201443f0 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5180be070 con 0x7fa5201443f0 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5200b1370 con 0x7fa5201443f0 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa518004040 con 0x7fa5201443f0 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201427e0 con 0x7fa5201443f0 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa520068970 con 0x7fa5201443f0 2024-04-25T00:56:42.522 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa5180041e0 con 0x7fa5201443f0 2024-04-25T00:56:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa50effd640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5000678c0 0x7fa500069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.514+0000 7fa51e59c640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5000678c0 0x7fa500069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.518+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fa518121570 con 0x7fa5201443f0 2024-04-25T00:56:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.518+0000 7fa50effd640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fa50006d780 0x7fa50006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.518+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 --> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5180b4e50 con 0x7fa50006d780 2024-04-25T00:56:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.518+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa518121a00 con 0x7fa5201443f0 2024-04-25T00:56:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.518+0000 7fa51e59c640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5000678c0 0x7fa500069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa510005890 tx=0x7fa510009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.523 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.518+0000 7fa51dd9b640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fa50006d780 0x7fa50006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.518+0000 7fa51dd9b640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fa50006d780 0x7fa50006fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.524 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.518+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 <== osd.4 v2:172.21.15.187:6801/1453523944 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5180b4e50 con 0x7fa50006d780 2024-04-25T00:56:42.534 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.530+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 --> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa5200b1090 con 0x7fa50006d780 2024-04-25T00:56:42.535 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa50effd640 1 -- 172.21.15.112:0/1148423066 <== osd.4 v2:172.21.15.187:6801/1453523944 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa5200b1090 con 0x7fa50006d780 2024-04-25T00:56:42.535 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:42.536 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:56:42.536 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fa50006d780 msgr2=0x7fa50006fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.187:6801/1453523944,v1:172.21.15.187:6803/1453523944] conn(0x7fa50006d780 0x7fa50006fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5000678c0 msgr2=0x7fa500069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fa5000678c0 0x7fa500069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa510005890 tx=0x7fa510009040 comp rx=0 tx=0).stop 2024-04-25T00:56:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 msgr2=0x7fa5201e31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.537 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 --2- 172.21.15.112:0/1148423066 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa5201443f0 0x7fa5201e31b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa5180b0040 tx=0x7fa518019680 comp rx=0 tx=0).stop 2024-04-25T00:56:42.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa51ed9d640 1 -- 172.21.15.112:0/1148423066 reap_dead start 2024-04-25T00:56:42.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 shutdown_connections 2024-04-25T00:56:42.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 >> 172.21.15.112:0/1148423066 conn(0x7fa5200b5c30 msgr2=0x7fa520145630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 shutdown_connections 2024-04-25T00:56:42.538 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.534+0000 7fa524fe6640 1 -- 172.21.15.112:0/1148423066 wait complete. 2024-04-25T00:56:42.550 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T00:56:42.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76ad5d3640 1 Processor -- start 2024-04-25T00:56:42.659 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76ad5d3640 1 -- start start 2024-04-25T00:56:42.660 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76ad5d3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f76a8074500 0x7f76a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76ad5d3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f76a80b9cb0 0x7f76a809b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76ad5d3640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76a80b43a0 con 0x7f76a80b58b0 2024-04-25T00:56:42.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76ad5d3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f76a80b3b90 con 0x7f76a80b9cb0 2024-04-25T00:56:42.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76ad5d3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f76a80ae830 con 0x7f76a8074500 2024-04-25T00:56:42.661 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f76a80b9cb0 0x7f76a809b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a77fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f76a80b9cb0 0x7f76a809b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32850/0 (socket says 172.21.15.112:32850) 2024-04-25T00:56:42.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a77fe640 1 -- 172.21.15.112:0/825747565 learned_addr learned my addr 172.21.15.112:0/825747565 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:42.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a6ffd640 1 --2- 172.21.15.112:0/825747565 >> v2:172.21.15.187:3300/0 conn(0x7f76a8074500 0x7f76a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a77fe640 1 -- 172.21.15.112:0/825747565 >> v2:172.21.15.187:3300/0 conn(0x7f76a8074500 msgr2=0x7f76a80b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:42.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a77fe640 1 --2- 172.21.15.112:0/825747565 >> v2:172.21.15.187:3300/0 conn(0x7f76a8074500 0x7f76a80b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a77fe640 1 -- 172.21.15.112:0/825747565 >> v1:172.21.15.187:6789/0 conn(0x7f76a80b58b0 legacy=0x7f76a809aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:42.662 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a77fe640 1 -- 172.21.15.112:0/825747565 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a80b3de0 con 0x7f76a80b9cb0 2024-04-25T00:56:42.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.658+0000 7f76a77fe640 1 --2- 172.21.15.112:0/825747565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b9cb0 0x7f76a809b140 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009e50 tx=0x7f76a001c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb09defe1dfa84b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76a67fc640 1 -- 172.21.15.112:0/825747565 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00a5740 con 0x7f76a80b9cb0 2024-04-25T00:56:42.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76a67fc640 1 -- 172.21.15.112:0/825747565 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a00a58e0 con 0x7f76a80b9cb0 2024-04-25T00:56:42.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76a67fc640 1 -- 172.21.15.112:0/825747565 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a00a5cb0 con 0x7f76a80b9cb0 2024-04-25T00:56:42.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- 172.21.15.112:0/825747565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b9cb0 msgr2=0x7f76a809b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/825747565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b9cb0 0x7f76a809b140 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009e50 tx=0x7f76a001c860 comp rx=0 tx=0).stop 2024-04-25T00:56:42.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- 172.21.15.112:0/825747565 shutdown_connections 2024-04-25T00:56:42.664 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/825747565 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b9cb0 0x7f76a809b140 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/825747565 >> v2:172.21.15.187:3300/0 conn(0x7f76a8074500 0x7f76a80b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- 172.21.15.112:0/825747565 >> 172.21.15.112:0/825747565 conn(0x7f76a80b62f0 msgr2=0x7f76a80b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- 172.21.15.112:0/825747565 shutdown_connections 2024-04-25T00:56:42.665 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- 172.21.15.112:0/825747565 wait complete. 2024-04-25T00:56:42.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 Processor -- start 2024-04-25T00:56:42.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- start start 2024-04-25T00:56:42.667 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76a8074500 0x7f76a80b3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b58b0 0x7f76a8070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76a80b9cb0 0x7f76a80b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f76a814acd0 con 0x7f76a80b58b0 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f76a814ab50 con 0x7f76a8074500 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76ad5d3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f76a809b680 con 0x7f76a80b9cb0 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f769ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b58b0 0x7f76a8070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f769ffff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b58b0 0x7f76a8070bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:32860/0 (socket says 172.21.15.112:32860) 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f769ffff640 1 -- 172.21.15.112:0/269643493 learned_addr learned my addr 172.21.15.112:0/269643493 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:42.668 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76a6ffd640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76a8074500 0x7f76a80b3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76a77fe640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76a80b9cb0 0x7f76a80b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f769ffff640 1 -- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76a80b9cb0 msgr2=0x7f76a80b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f769ffff640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76a80b9cb0 0x7f76a80b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f769ffff640 1 -- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76a8074500 msgr2=0x7f76a80b3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f769ffff640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76a8074500 0x7f76a80b3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f769ffff640 1 -- 172.21.15.112:0/269643493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a809c490 con 0x7f76a80b58b0 2024-04-25T00:56:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.662+0000 7f76a77fe640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76a80b9cb0 0x7f76a80b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:42.669 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f769ffff640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b58b0 0x7f76a8070bd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f769000c400 tx=0x7f769000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f769f7fe640 1 -- 172.21.15.112:0/269643493 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690007020 con 0x7f76a80b58b0 2024-04-25T00:56:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f769f7fe640 1 -- 172.21.15.112:0/269643493 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76900071c0 con 0x7f76a80b58b0 2024-04-25T00:56:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a809c770 con 0x7f76a80b58b0 2024-04-25T00:56:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f769f7fe640 1 -- 172.21.15.112:0/269643493 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76900075b0 con 0x7f76a80b58b0 2024-04-25T00:56:42.670 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81e0230 con 0x7f76a80b58b0 2024-04-25T00:56:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a804f510 con 0x7f76a80b58b0 2024-04-25T00:56:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f769f7fe640 1 -- 172.21.15.112:0/269643493 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7690019020 con 0x7f76a80b58b0 2024-04-25T00:56:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f769f7fe640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f768c0678c0 0x7f768c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f769f7fe640 1 -- 172.21.15.112:0/269643493 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f7690085170 con 0x7f76a80b58b0 2024-04-25T00:56:42.671 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f76a6ffd640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f768c0678c0 0x7f768c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.672 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.666+0000 7f76a6ffd640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f768c0678c0 0x7f768c069d80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f769800b930 tx=0x7f7698008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.673 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.670+0000 7f769f7fe640 1 -- 172.21.15.112:0/269643493 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7690055fc0 con 0x7f76a80b58b0 2024-04-25T00:56:42.822 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.818+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f76a8148af0 con 0x7f768c0678c0 2024-04-25T00:56:42.823 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f769f7fe640 1 -- 172.21.15.112:0/269643493 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f76a8148af0 con 0x7f768c0678c0 2024-04-25T00:56:42.823 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T00:56:42.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f768c0678c0 msgr2=0x7f768c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.825 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f768c0678c0 0x7f768c069d80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f769800b930 tx=0x7f7698008040 comp rx=0 tx=0).stop 2024-04-25T00:56:42.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b58b0 msgr2=0x7f76a8070bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b58b0 0x7f76a8070bd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f769000c400 tx=0x7f769000c8c0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 shutdown_connections 2024-04-25T00:56:42.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f76a80b9cb0 0x7f76a80b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f76a80b58b0 0x7f76a8070bd0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f768c0678c0 0x7f768c069d80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 --2- 172.21.15.112:0/269643493 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f76a8074500 0x7f76a80b3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.826 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 >> 172.21.15.112:0/269643493 conn(0x7f76a80b62f0 msgr2=0x7f76a809aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 shutdown_connections 2024-04-25T00:56:42.827 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.822+0000 7f76ad5d3640 1 -- 172.21.15.112:0/269643493 wait complete. 2024-04-25T00:56:42.840 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T00:56:42.841 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:56:42.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6999f3640 1 Processor -- start 2024-04-25T00:56:42.952 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6999f3640 1 -- start start 2024-04-25T00:56:42.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6999f3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc6940b21e0 0x7fc694148700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6999f3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6940b4d90 0x7fc69413ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6999f3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6940b25c0 con 0x7fc69413f4a0 2024-04-25T00:56:42.953 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6999f3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fc69413f920 con 0x7fc6940b21e0 2024-04-25T00:56:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6999f3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc69406f100 con 0x7fc6940b4d90 2024-04-25T00:56:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6937fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fc69413f4a0 0x7fc694145150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36868/0 (socket says 172.21.15.112:36868) 2024-04-25T00:56:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6937fe640 1 -- 172.21.15.112:0/918805493 learned_addr learned my addr 172.21.15.112:0/918805493 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc692ffd640 1 --2- 172.21.15.112:0/918805493 >> v2:172.21.15.112:3300/0 conn(0x7fc6940b21e0 0x7fc694148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc6927fc640 1 --2- 172.21.15.112:0/918805493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6940b4d90 0x7fc69413ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc692ffd640 1 -- 172.21.15.112:0/918805493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6940b4d90 msgr2=0x7fc69413ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc692ffd640 1 --2- 172.21.15.112:0/918805493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6940b4d90 0x7fc69413ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.954 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc692ffd640 1 -- 172.21.15.112:0/918805493 >> v1:172.21.15.112:6789/0 conn(0x7fc69413f4a0 legacy=0x7fc694145150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc691ffb640 1 -- 172.21.15.112:0/918805493 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2726604147 0 0) 0x7fc6940b25c0 con 0x7fc69413f4a0 2024-04-25T00:56:42.955 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.950+0000 7fc692ffd640 1 -- 172.21.15.112:0/918805493 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69406f630 con 0x7fc6940b21e0 2024-04-25T00:56:42.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc692ffd640 1 --2- 172.21.15.112:0/918805493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6940b21e0 0x7fc694148700 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009d90 tx=0x7fc68c01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1645e20432dbfb4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc691ffb640 1 -- 172.21.15.112:0/918805493 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0b2070 con 0x7fc6940b21e0 2024-04-25T00:56:42.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc691ffb640 1 -- 172.21.15.112:0/918805493 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c0a5880 con 0x7fc6940b21e0 2024-04-25T00:56:42.956 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc691ffb640 1 -- 172.21.15.112:0/918805493 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0a5c50 con 0x7fc6940b21e0 2024-04-25T00:56:42.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- 172.21.15.112:0/918805493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6940b21e0 msgr2=0x7fc694148700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 --2- 172.21.15.112:0/918805493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6940b21e0 0x7fc694148700 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc68c009d90 tx=0x7fc68c01c820 comp rx=0 tx=0).stop 2024-04-25T00:56:42.957 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- 172.21.15.112:0/918805493 shutdown_connections 2024-04-25T00:56:42.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 --2- 172.21.15.112:0/918805493 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6940b4d90 0x7fc69413ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 --2- 172.21.15.112:0/918805493 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6940b21e0 0x7fc694148700 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- 172.21.15.112:0/918805493 >> 172.21.15.112:0/918805493 conn(0x7fc6940b8930 msgr2=0x7fc6940b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- 172.21.15.112:0/918805493 shutdown_connections 2024-04-25T00:56:42.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- 172.21.15.112:0/918805493 wait complete. 2024-04-25T00:56:42.958 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 Processor -- start 2024-04-25T00:56:42.959 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- start start 2024-04-25T00:56:42.961 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6940b21e0 0x7fc6941423a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69413f4a0 0x7fc694138010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6941428e0 0x7fc694138550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fc69406d790 con 0x7fc6941428e0 2024-04-25T00:56:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fc694148c40 con 0x7fc69413f4a0 2024-04-25T00:56:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6999f3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fc6940b4990 con 0x7fc6940b21e0 2024-04-25T00:56:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6937fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6941428e0 0x7fc694138550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6937fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6941428e0 0x7fc694138550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46220/0 (socket says 172.21.15.112:46220) 2024-04-25T00:56:42.962 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6937fe640 1 -- 172.21.15.112:0/132329455 learned_addr learned my addr 172.21.15.112:0/132329455 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:42.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.954+0000 7fc6927fc640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69413f4a0 0x7fc694138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6937fe640 1 -- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6940b21e0 msgr2=0x7fc6941423a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6937fe640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fc6940b21e0 0x7fc6941423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6937fe640 1 -- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69413f4a0 msgr2=0x7fc694138010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6937fe640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69413f4a0 0x7fc694138010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.963 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6937fe640 1 -- 172.21.15.112:0/132329455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc694141940 con 0x7fc6941428e0 2024-04-25T00:56:42.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6927fc640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fc69413f4a0 0x7fc694138010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:42.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6937fe640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6941428e0 0x7fc694138550 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c3e0 tx=0x7fc68400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6840074e0 con 0x7fc6941428e0 2024-04-25T00:56:42.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6840047f0 con 0x7fc6941428e0 2024-04-25T00:56:42.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684004be0 con 0x7fc6941428e0 2024-04-25T00:56:42.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc694137350 con 0x7fc6941428e0 2024-04-25T00:56:42.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc694140630 con 0x7fc6941428e0 2024-04-25T00:56:42.964 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc694068260 con 0x7fc6941428e0 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc684011070 con 0x7fc6941428e0 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc6800678f0 0x7fc680069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fc684085d30 con 0x7fc6941428e0 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc692ffd640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc6800678f0 0x7fc680069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7fc68006d7b0 0x7fc68006fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 --> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc68400bf30 con 0x7fc68006d7b0 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc6840861c0 con 0x7fc6941428e0 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc692ffd640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc6800678f0 0x7fc680069db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0af000 tx=0x7fc68c004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.965 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6927fc640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7fc68006d7b0 0x7fc68006fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:42.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.958+0000 7fc6927fc640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7fc68006d7b0 0x7fc68006fbd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:42.966 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.962+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 <== osd.5 v2:172.21.15.187:6816/2456695186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc68400bf30 con 0x7fc68006d7b0 2024-04-25T00:56:42.976 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 --> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc6940b67a0 con 0x7fc68006d7b0 2024-04-25T00:56:42.977 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc67bfff640 1 -- 172.21.15.112:0/132329455 <== osd.5 v2:172.21.15.187:6816/2456695186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc6940b67a0 con 0x7fc68006d7b0 2024-04-25T00:56:42.977 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:42.977 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:56:42.977 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:42.978 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7fc68006d7b0 msgr2=0x7fc68006fbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.187:6816/2456695186,v1:172.21.15.187:6817/2456695186] conn(0x7fc68006d7b0 0x7fc68006fbd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.979 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc6800678f0 msgr2=0x7fc680069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fc6800678f0 0x7fc680069db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0af000 tx=0x7fc68c004ac0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6941428e0 msgr2=0x7fc694138550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:42.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 --2- 172.21.15.112:0/132329455 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc6941428e0 0x7fc694138550 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc68400c3e0 tx=0x7fc68400c8a0 comp rx=0 tx=0).stop 2024-04-25T00:56:42.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6937fe640 1 -- 172.21.15.112:0/132329455 reap_dead start 2024-04-25T00:56:42.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 shutdown_connections 2024-04-25T00:56:42.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 >> 172.21.15.112:0/132329455 conn(0x7fc6940b8930 msgr2=0x7fc6940b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:42.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 shutdown_connections 2024-04-25T00:56:42.980 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:42.974+0000 7fc6999f3640 1 -- 172.21.15.112:0/132329455 wait complete. 2024-04-25T00:56:42.992 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T00:56:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.098+0000 7f665c844640 1 Processor -- start 2024-04-25T00:56:43.102 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.098+0000 7f665c844640 1 -- start start 2024-04-25T00:56:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.098+0000 7f665c844640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f66580b6120 0x7f665813e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.103 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.098+0000 7f665c844640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66580b59e0 0x7f665814e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.098+0000 7f665c844640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66580b6a00 con 0x7f665813e5f0 2024-04-25T00:56:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.098+0000 7f665c844640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f6658072bd0 con 0x7f66580b59e0 2024-04-25T00:56:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.098+0000 7f665c844640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f6658072d50 con 0x7f66580b6120 2024-04-25T00:56:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f6656d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66580b59e0 0x7f665814e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f6656d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66580b59e0 0x7f665814e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46224/0 (socket says 172.21.15.112:46224) 2024-04-25T00:56:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f6656d9d640 1 -- 172.21.15.112:0/3844428969 learned_addr learned my addr 172.21.15.112:0/3844428969 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665659c640 1 --2- 172.21.15.112:0/3844428969 >> v2:172.21.15.187:3300/0 conn(0x7f66580b6120 0x7f665813e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.104 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f6656d9d640 1 -- 172.21.15.112:0/3844428969 >> v2:172.21.15.187:3300/0 conn(0x7f66580b6120 msgr2=0x7f665813e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f6656d9d640 1 --2- 172.21.15.112:0/3844428969 >> v2:172.21.15.187:3300/0 conn(0x7f66580b6120 0x7f665813e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f6656d9d640 1 -- 172.21.15.112:0/3844428969 >> v1:172.21.15.187:6789/0 conn(0x7f665813e5f0 legacy=0x7f6658144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.105 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f6656d9d640 1 -- 172.21.15.112:0/3844428969 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6658144070 con 0x7f66580b59e0 2024-04-25T00:56:43.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f6656d9d640 1 --2- 172.21.15.112:0/3844428969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b59e0 0x7f665814e9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f66580b7850 tx=0x7f665001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f1e494e7e46caf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665559a640 1 -- 172.21.15.112:0/3844428969 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500a57c0 con 0x7f66580b59e0 2024-04-25T00:56:43.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665559a640 1 -- 172.21.15.112:0/3844428969 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66500a5960 con 0x7f66580b59e0 2024-04-25T00:56:43.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665559a640 1 -- 172.21.15.112:0/3844428969 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500a5d30 con 0x7f66580b59e0 2024-04-25T00:56:43.106 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665c844640 1 -- 172.21.15.112:0/3844428969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b59e0 msgr2=0x7f665814e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665c844640 1 --2- 172.21.15.112:0/3844428969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b59e0 0x7f665814e9c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f66580b7850 tx=0x7f665001c8e0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665c844640 1 -- 172.21.15.112:0/3844428969 shutdown_connections 2024-04-25T00:56:43.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665c844640 1 --2- 172.21.15.112:0/3844428969 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b59e0 0x7f665814e9c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665c844640 1 --2- 172.21.15.112:0/3844428969 >> v2:172.21.15.187:3300/0 conn(0x7f66580b6120 0x7f665813e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665c844640 1 -- 172.21.15.112:0/3844428969 >> 172.21.15.112:0/3844428969 conn(0x7f665806cf00 msgr2=0x7f665806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:43.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665c844640 1 -- 172.21.15.112:0/3844428969 shutdown_connections 2024-04-25T00:56:43.107 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.102+0000 7f665c844640 1 -- 172.21.15.112:0/3844428969 wait complete. 2024-04-25T00:56:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 Processor -- start 2024-04-25T00:56:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 -- start start 2024-04-25T00:56:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66580b59e0 0x7f665813ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b6120 0x7f665813f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f665813e5f0 0x7f66581e1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f6658151ab0 con 0x7f66580b6120 2024-04-25T00:56:43.110 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f6658151930 con 0x7f665813e5f0 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f6658151c30 con 0x7f66580b59e0 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b6120 0x7f665813f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b6120 0x7f665813f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46230/0 (socket says 172.21.15.112:46230) 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 -- 172.21.15.112:0/1816604075 learned_addr learned my addr 172.21.15.112:0/1816604075 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665659c640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66580b59e0 0x7f665813ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6656d9d640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f665813e5f0 0x7f66581e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 -- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66580b59e0 msgr2=0x7f665813ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66580b59e0 0x7f665813ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.111 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 -- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f665813e5f0 msgr2=0x7f66581e1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f665813e5f0 0x7f66581e1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 -- 172.21.15.112:0/1816604075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66580a94a0 con 0x7f66580b6120 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6656d9d640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f665813e5f0 0x7f66581e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6655d9b640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b6120 0x7f665813f4d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f664c00b410 tx=0x7f664c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6646ffd640 1 -- 172.21.15.112:0/1816604075 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c015030 con 0x7f66580b6120 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6646ffd640 1 -- 172.21.15.112:0/1816604075 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f664c004020 con 0x7f66580b6120 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665813e9f0 con 0x7f66580b6120 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6646ffd640 1 -- 172.21.15.112:0/1816604075 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f664c004450 con 0x7f66580b6120 2024-04-25T00:56:43.112 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66580aa3f0 con 0x7f66580b6120 2024-04-25T00:56:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665804f580 con 0x7f66580b6120 2024-04-25T00:56:43.113 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.106+0000 7f6646ffd640 1 -- 172.21.15.112:0/1816604075 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f664c018070 con 0x7f66580b6120 2024-04-25T00:56:43.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.110+0000 7f6646ffd640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66380678f0 0x7f6638069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.110+0000 7f6646ffd640 1 -- 172.21.15.112:0/1816604075 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f664c0854e0 con 0x7f66580b6120 2024-04-25T00:56:43.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.110+0000 7f665659c640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66380678f0 0x7f6638069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.114 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.110+0000 7f665659c640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66380678f0 0x7f6638069db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f664800b9d0 tx=0x7f6648008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.115 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.114+0000 7f6646ffd640 1 -- 172.21.15.112:0/1816604075 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f664c056320 con 0x7f66580b6120 2024-04-25T00:56:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f665813efb0 con 0x7f66380678f0 2024-04-25T00:56:43.263 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f6646ffd640 1 -- 172.21.15.112:0/1816604075 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f665813efb0 con 0x7f66380678f0 2024-04-25T00:56:43.264 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T00:56:43.265 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66380678f0 msgr2=0x7f6638069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66380678f0 0x7f6638069db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f664800b9d0 tx=0x7f6648008040 comp rx=0 tx=0).stop 2024-04-25T00:56:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b6120 msgr2=0x7f665813f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b6120 0x7f665813f4d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f664c00b410 tx=0x7f664c00b8d0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 shutdown_connections 2024-04-25T00:56:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f66380678f0 0x7f6638069db0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.266 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f665813e5f0 0x7f66581e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66580b6120 0x7f665813f4d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 --2- 172.21.15.112:0/1816604075 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f66580b59e0 0x7f665813ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 >> 172.21.15.112:0/1816604075 conn(0x7f665806cf00 msgr2=0x7f66580ba0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 shutdown_connections 2024-04-25T00:56:43.267 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.262+0000 7f665c844640 1 -- 172.21.15.112:0/1816604075 wait complete. 2024-04-25T00:56:43.281 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T00:56:43.281 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:56:43.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5698cf640 1 Processor -- start 2024-04-25T00:56:43.400 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5698cf640 1 -- start start 2024-04-25T00:56:43.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5698cf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5640b4de0 0x7ff56406f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5698cf640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7ff5641487f0 0x7ff564148c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.401 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5698cf640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff564074a70 con 0x7ff564074690 2024-04-25T00:56:43.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5698cf640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7ff5640b3030 con 0x7ff5640b4de0 2024-04-25T00:56:43.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5698cf640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7ff564070fe0 con 0x7ff5641487f0 2024-04-25T00:56:43.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5627fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5640b4de0 0x7ff56406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5627fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5640b4de0 0x7ff56406f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46246/0 (socket says 172.21.15.112:46246) 2024-04-25T00:56:43.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5627fc640 1 -- 172.21.15.112:0/1300347302 learned_addr learned my addr 172.21.15.112:0/1300347302 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:43.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5627fc640 1 -- 172.21.15.112:0/1300347302 >> v2:172.21.15.187:3300/0 conn(0x7ff5641487f0 msgr2=0x7ff564148c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.402 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5627fc640 1 --2- 172.21.15.112:0/1300347302 >> v2:172.21.15.187:3300/0 conn(0x7ff5641487f0 0x7ff564148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5627fc640 1 -- 172.21.15.112:0/1300347302 >> v1:172.21.15.187:6789/0 conn(0x7ff564074690 legacy=0x7ff56413f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.403 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.398+0000 7ff5627fc640 1 -- 172.21.15.112:0/1300347302 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff564149140 con 0x7ff5640b4de0 2024-04-25T00:56:43.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5627fc640 1 --2- 172.21.15.112:0/1300347302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5640b4de0 0x7ff56406f950 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff564074fe0 tx=0x7ff55401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bf5653aa7e61bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff561ffb640 1 -- 172.21.15.112:0/1300347302 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540b0070 con 0x7ff5640b4de0 2024-04-25T00:56:43.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff561ffb640 1 -- 172.21.15.112:0/1300347302 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5540ab600 con 0x7ff5640b4de0 2024-04-25T00:56:43.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff561ffb640 1 -- 172.21.15.112:0/1300347302 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540ab9d0 con 0x7ff5640b4de0 2024-04-25T00:56:43.404 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- 172.21.15.112:0/1300347302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5640b4de0 msgr2=0x7ff56406f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 --2- 172.21.15.112:0/1300347302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5640b4de0 0x7ff56406f950 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff564074fe0 tx=0x7ff55401c450 comp rx=0 tx=0).stop 2024-04-25T00:56:43.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- 172.21.15.112:0/1300347302 shutdown_connections 2024-04-25T00:56:43.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 --2- 172.21.15.112:0/1300347302 >> v2:172.21.15.187:3300/0 conn(0x7ff5641487f0 0x7ff564148c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 --2- 172.21.15.112:0/1300347302 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5640b4de0 0x7ff56406f950 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- 172.21.15.112:0/1300347302 >> 172.21.15.112:0/1300347302 conn(0x7ff5640b5c30 msgr2=0x7ff5640b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:43.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- 172.21.15.112:0/1300347302 shutdown_connections 2024-04-25T00:56:43.405 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- 172.21.15.112:0/1300347302 wait complete. 2024-04-25T00:56:43.407 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 Processor -- start 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- start start 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff564074690 0x7ff56413de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5640b4de0 0x7ff564147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5641487f0 0x7ff564147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7ff564152540 con 0x7ff5641487f0 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7ff56413f980 con 0x7ff5640b4de0 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5698cf640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7ff5641526c0 con 0x7ff564074690 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5641487f0 0x7ff564147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.408 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5637fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5641487f0 0x7ff564147650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46254/0 (socket says 172.21.15.112:46254) 2024-04-25T00:56:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5637fe640 1 -- 172.21.15.112:0/2727911950 learned_addr learned my addr 172.21.15.112:0/2727911950 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5627fc640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5640b4de0 0x7ff564147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff562ffd640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff564074690 0x7ff56413de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5637fe640 1 -- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff564074690 msgr2=0x7ff56413de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5637fe640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7ff564074690 0x7ff56413de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5637fe640 1 -- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5640b4de0 msgr2=0x7ff564147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5637fe640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7ff5640b4de0 0x7ff564147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.409 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.402+0000 7ff5637fe640 1 -- 172.21.15.112:0/2727911950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff564073430 con 0x7ff5641487f0 2024-04-25T00:56:43.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff5637fe640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5641487f0 0x7ff564147650 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00b970 tx=0x7ff55c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c019070 con 0x7ff5641487f0 2024-04-25T00:56:43.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c00cb60 con 0x7ff5641487f0 2024-04-25T00:56:43.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff564073150 con 0x7ff5641487f0 2024-04-25T00:56:43.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c014440 con 0x7ff5641487f0 2024-04-25T00:56:43.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56413caa0 con 0x7ff5641487f0 2024-04-25T00:56:43.410 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff564148220 con 0x7ff5641487f0 2024-04-25T00:56:43.411 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff55c01c070 con 0x7ff5641487f0 2024-04-25T00:56:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ff5500678c0 0x7ff550069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7ff55c085ce0 con 0x7ff5641487f0 2024-04-25T00:56:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff562ffd640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ff5500678c0 0x7ff550069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7ff55006d780 0x7ff55006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 --> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff550070250 con 0x7ff55006d780 2024-04-25T00:56:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff55c086170 con 0x7ff5641487f0 2024-04-25T00:56:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff562ffd640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ff5500678c0 0x7ff550069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff55800b810 tx=0x7ff558008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.412 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff5627fc640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7ff55006d780 0x7ff55006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.406+0000 7ff5627fc640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7ff55006d780 0x7ff55006fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.413 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.410+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 <== osd.6 v2:172.21.15.187:6800/718817690 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff550070250 con 0x7ff55006d780 2024-04-25T00:56:43.423 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 --> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff5640b8d20 con 0x7ff55006d780 2024-04-25T00:56:43.424 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff54bfff640 1 -- 172.21.15.112:0/2727911950 <== osd.6 v2:172.21.15.187:6800/718817690 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff5640b8d20 con 0x7ff55006d780 2024-04-25T00:56:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:43.425 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:56:43.425 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7ff55006d780 msgr2=0x7ff55006fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.187:6800/718817690,v1:172.21.15.187:6802/718817690] conn(0x7ff55006d780 0x7ff55006fba0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ff5500678c0 msgr2=0x7ff550069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7ff5500678c0 0x7ff550069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff55800b810 tx=0x7ff558008040 comp rx=0 tx=0).stop 2024-04-25T00:56:43.426 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5641487f0 msgr2=0x7ff564147650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 --2- 172.21.15.112:0/2727911950 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5641487f0 0x7ff564147650 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00b970 tx=0x7ff55c00be30 comp rx=0 tx=0).stop 2024-04-25T00:56:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5637fe640 1 -- 172.21.15.112:0/2727911950 reap_dead start 2024-04-25T00:56:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 shutdown_connections 2024-04-25T00:56:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 >> 172.21.15.112:0/2727911950 conn(0x7ff5640b5c30 msgr2=0x7ff564069090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 shutdown_connections 2024-04-25T00:56:43.427 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.422+0000 7ff5698cf640 1 -- 172.21.15.112:0/2727911950 wait complete. 2024-04-25T00:56:43.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T00:56:43.551 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd293b7640 1 Processor -- start 2024-04-25T00:56:43.552 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd293b7640 1 -- start start 2024-04-25T00:56:43.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd293b7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdd240722d0 0x7fdd24072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd293b7640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdd2409c270 0x7fdd24147d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd293b7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd240bbec0 con 0x7fdd24147650 2024-04-25T00:56:43.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd293b7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fdd240baaf0 con 0x7fdd240722d0 2024-04-25T00:56:43.553 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd293b7640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdd24073100 con 0x7fdd2409c270 2024-04-25T00:56:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd22ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdd240722d0 0x7fdd24072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd237fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fdd24147650 0x7fdd2414ab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36874/0 (socket says 172.21.15.112:36874) 2024-04-25T00:56:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd237fe640 1 -- 172.21.15.112:0/2723485541 learned_addr learned my addr 172.21.15.112:0/2723485541 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd227fc640 1 --2- 172.21.15.112:0/2723485541 >> v2:172.21.15.187:3300/0 conn(0x7fdd2409c270 0x7fdd24147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd22ffd640 1 -- 172.21.15.112:0/2723485541 >> v2:172.21.15.187:3300/0 conn(0x7fdd2409c270 msgr2=0x7fdd24147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd22ffd640 1 --2- 172.21.15.112:0/2723485541 >> v2:172.21.15.187:3300/0 conn(0x7fdd2409c270 0x7fdd24147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd22ffd640 1 -- 172.21.15.112:0/2723485541 >> v1:172.21.15.112:6789/0 conn(0x7fdd24147650 legacy=0x7fdd2414ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.554 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.550+0000 7fdd22ffd640 1 -- 172.21.15.112:0/2723485541 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd240bc5c0 con 0x7fdd240722d0 2024-04-25T00:56:43.555 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd22ffd640 1 --2- 172.21.15.112:0/2723485541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd240722d0 0x7fdd24072bc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdd240ba460 tx=0x7fdd0c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d0bf4d023e77de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd21ffb640 1 -- 172.21.15.112:0/2723485541 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0b1070 con 0x7fdd240722d0 2024-04-25T00:56:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd21ffb640 1 -- 172.21.15.112:0/2723485541 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd0c0ac620 con 0x7fdd240722d0 2024-04-25T00:56:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd21ffb640 1 -- 172.21.15.112:0/2723485541 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0ac9f0 con 0x7fdd240722d0 2024-04-25T00:56:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- 172.21.15.112:0/2723485541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd240722d0 msgr2=0x7fdd24072bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2723485541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd240722d0 0x7fdd24072bc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdd240ba460 tx=0x7fdd0c01c3c0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- 172.21.15.112:0/2723485541 shutdown_connections 2024-04-25T00:56:43.556 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2723485541 >> v2:172.21.15.187:3300/0 conn(0x7fdd2409c270 0x7fdd24147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2723485541 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd240722d0 0x7fdd24072bc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- 172.21.15.112:0/2723485541 >> 172.21.15.112:0/2723485541 conn(0x7fdd2409e8c0 msgr2=0x7fdd2409ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:43.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- 172.21.15.112:0/2723485541 shutdown_connections 2024-04-25T00:56:43.557 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- 172.21.15.112:0/2723485541 wait complete. 2024-04-25T00:56:43.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 Processor -- start 2024-04-25T00:56:43.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- start start 2024-04-25T00:56:43.561 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd240722d0 0x7fdd24143c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd2409c270 0x7fdd2413ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd24147650 0x7fdd2413de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24154f10 con 0x7fdd2409c270 2024-04-25T00:56:43.562 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2409ac50 con 0x7fdd240722d0 2024-04-25T00:56:43.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdd24155090 con 0x7fdd24147650 2024-04-25T00:56:43.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd2409c270 0x7fdd2413ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd2409c270 0x7fdd2413ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46270/0 (socket says 172.21.15.112:46270) 2024-04-25T00:56:43.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 -- 172.21.15.112:0/2458366602 learned_addr learned my addr 172.21.15.112:0/2458366602 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:43.563 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd22ffd640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd240722d0 0x7fdd24143c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 -- 172.21.15.112:0/2458366602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd24147650 msgr2=0x7fdd2413de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd24147650 0x7fdd2413de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 -- 172.21.15.112:0/2458366602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd240722d0 msgr2=0x7fdd24143c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd240722d0 0x7fdd24143c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.564 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 -- 172.21.15.112:0/2458366602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd24144a40 con 0x7fdd2409c270 2024-04-25T00:56:43.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd22ffd640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd240722d0 0x7fdd24143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:43.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd227fc640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd2409c270 0x7fdd2413ffc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400d800 tx=0x7fdd1400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd1b7fe640 1 -- 172.21.15.112:0/2458366602 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400b6c0 con 0x7fdd2409c270 2024-04-25T00:56:43.565 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd1b7fe640 1 -- 172.21.15.112:0/2458366602 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd14019070 con 0x7fdd2409c270 2024-04-25T00:56:43.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd2413e8a0 con 0x7fdd2409c270 2024-04-25T00:56:43.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd1b7fe640 1 -- 172.21.15.112:0/2458366602 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400ba60 con 0x7fdd2409c270 2024-04-25T00:56:43.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.554+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd24141a50 con 0x7fdd2409c270 2024-04-25T00:56:43.566 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.558+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd240b3530 con 0x7fdd2409c270 2024-04-25T00:56:43.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.558+0000 7fdd1b7fe640 1 -- 172.21.15.112:0/2458366602 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdd1400bc00 con 0x7fdd2409c270 2024-04-25T00:56:43.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.558+0000 7fdd1b7fe640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdd040678c0 0x7fdd04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.558+0000 7fdd1b7fe640 1 -- 172.21.15.112:0/2458366602 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fdd14085e60 con 0x7fdd2409c270 2024-04-25T00:56:43.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.558+0000 7fdd22ffd640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdd040678c0 0x7fdd04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.567 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.558+0000 7fdd22ffd640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdd040678c0 0x7fdd04069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ae000 tx=0x7fdd0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.568 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.562+0000 7fdd1b7fe640 1 -- 172.21.15.112:0/2458366602 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdd14056c90 con 0x7fdd2409c270 2024-04-25T00:56:43.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.710+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fdd2404f580 con 0x7fdd040678c0 2024-04-25T00:56:43.714 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.710+0000 7fdd1b7fe640 1 -- 172.21.15.112:0/2458366602 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdd2404f580 con 0x7fdd040678c0 2024-04-25T00:56:43.715 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T00:56:43.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdd040678c0 msgr2=0x7fdd04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdd040678c0 0x7fdd04069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0ae000 tx=0x7fdd0c008040 comp rx=0 tx=0).stop 2024-04-25T00:56:43.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd2409c270 msgr2=0x7fdd2413ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd2409c270 0x7fdd2413ffc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400d800 tx=0x7fdd1400dcc0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.717 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 shutdown_connections 2024-04-25T00:56:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdd24147650 0x7fdd2413de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdd2409c270 0x7fdd2413ffc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdd040678c0 0x7fdd04069d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 --2- 172.21.15.112:0/2458366602 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdd240722d0 0x7fdd24143c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 >> 172.21.15.112:0/2458366602 conn(0x7fdd2409e8c0 msgr2=0x7fdd241421e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 shutdown_connections 2024-04-25T00:56:43.718 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.714+0000 7fdd293b7640 1 -- 172.21.15.112:0/2458366602 wait complete. 2024-04-25T00:56:43.731 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T00:56:43.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T00:56:43.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9af0ee3640 1 Processor -- start 2024-04-25T00:56:43.848 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9af0ee3640 1 -- start start 2024-04-25T00:56:43.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9af0ee3640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7f9aec0b4d90 0x7f9aec13ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9af0ee3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec13f4a0 0x7f9aec14fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9af0ee3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aec0b25c0 con 0x7f9aec0b21e0 2024-04-25T00:56:43.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9af0ee3640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7f9aec06f100 con 0x7f9aec0b4d90 2024-04-25T00:56:43.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9af0ee3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9aec06f630 con 0x7f9aec13f4a0 2024-04-25T00:56:43.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aea59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9aec0b21e0 0x7f9aec148700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36882/0 (socket says 172.21.15.112:36882) 2024-04-25T00:56:43.853 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aea59c640 1 -- 172.21.15.112:0/1696866240 learned_addr learned my addr 172.21.15.112:0/1696866240 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:43.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1336336425 0 0) 0x7f9aec0b25c0 con 0x7f9aec0b21e0 2024-04-25T00:56:43.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae9d9b640 1 --2- 172.21.15.112:0/1696866240 >> v2:172.21.15.187:3300/0 conn(0x7f9aec0b4d90 0x7f9aec13ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ad4003530 con 0x7f9aec0b21e0 2024-04-25T00:56:43.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aead9d640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec13f4a0 0x7f9aec14fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1535626385 0 0) 0x7f9ad4003530 con 0x7f9aec0b21e0 2024-04-25T00:56:43.854 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec13f4a0 msgr2=0x7f9aec14fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec13f4a0 0x7f9aec14fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 >> v2:172.21.15.187:3300/0 conn(0x7f9aec0b4d90 msgr2=0x7f9aec13ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 --2- 172.21.15.112:0/1696866240 >> v2:172.21.15.187:3300/0 conn(0x7f9aec0b4d90 0x7f9aec13ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec06f350 con 0x7f9aec0b21e0 2024-04-25T00:56:43.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f9adc001680 con 0x7f9aec0b21e0 2024-04-25T00:56:43.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 >> v1:172.21.15.112:6789/0 conn(0x7f9aec0b21e0 legacy=0x7f9aec148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.855 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad40039c0 0x7f9ad4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec13f4a0 0x7f9ad4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad4004d50 0x7f9ad40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec06f100 con 0x7f9ad40039c0 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec06f630 con 0x7f9ad4004d50 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad400bbe0 con 0x7f9aec13f4a0 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9ae9d9b640 1 --2- 172.21.15.112:0/1696866240 >> v2:172.21.15.187:3300/0 conn(0x7f9aec0b4d90 0x7f9aec13ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aea59c640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad40039c0 0x7f9ad4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aea59c640 1 -- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec13f4a0 msgr2=0x7f9ad4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.856 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aea59c640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec13f4a0 0x7f9ad4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aea59c640 1 -- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad4004d50 msgr2=0x7f9ad40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:43.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aea59c640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9ad4004d50 0x7f9ad40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.846+0000 7f9aea59c640 1 -- 172.21.15.112:0/1696866240 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad4003530 con 0x7f9ad40039c0 2024-04-25T00:56:43.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad40039c0 0x7f9ad4003de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c240 tx=0x7f9adc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf7603d95c1fbb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc007d40 con 0x7f9ad40039c0 2024-04-25T00:56:43.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9adc0a9020 con 0x7f9ad40039c0 2024-04-25T00:56:43.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9ae959a640 1 -- 172.21.15.112:0/1696866240 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc005ae0 con 0x7f9ad40039c0 2024-04-25T00:56:43.857 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1696866240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad40039c0 msgr2=0x7f9ad4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 --2- 172.21.15.112:0/1696866240 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9ad40039c0 0x7f9ad4003de0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c240 tx=0x7f9adc01cca0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aead9d640 1 -- 172.21.15.112:0/1696866240 reap_dead start 2024-04-25T00:56:43.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1696866240 shutdown_connections 2024-04-25T00:56:43.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1696866240 >> 172.21.15.112:0/1696866240 conn(0x7f9aec0b8930 msgr2=0x7f9aec0b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:43.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1696866240 shutdown_connections 2024-04-25T00:56:43.858 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1696866240 wait complete. 2024-04-25T00:56:43.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 Processor -- start 2024-04-25T00:56:43.861 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- start start 2024-04-25T00:56:43.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec072080 0x7f9aec0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9aec1e2f40 0x7f9aec1e33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec0ad8b0 0x7f9aec0add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec0b4b10 con 0x7f9aec072080 2024-04-25T00:56:43.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec0b4990 con 0x7f9aec1e2f40 2024-04-25T00:56:43.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9af0ee3640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f9aec1502a0 con 0x7f9aec0ad8b0 2024-04-25T00:56:43.862 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec072080 0x7f9aec0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec072080 0x7f9aec0b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46294/0 (socket says 172.21.15.112:46294) 2024-04-25T00:56:43.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 -- 172.21.15.112:0/1043384857 learned_addr learned my addr 172.21.15.112:0/1043384857 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:43.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9ae9d9b640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9aec1e2f40 0x7f9aec1e33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 -- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec0ad8b0 msgr2=0x7f9aec0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f9aec0ad8b0 0x7f9aec0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 -- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9aec1e2f40 msgr2=0x7f9aec1e33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f9aec1e2f40 0x7f9aec1e33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.863 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 -- 172.21.15.112:0/1043384857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec1e3900 con 0x7f9aec072080 2024-04-25T00:56:43.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9aea59c640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec072080 0x7f9aec0b1670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9adc001980 tx=0x7f9ad4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.850+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc001c80 con 0x7f9aec072080 2024-04-25T00:56:43.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9adc0b2070 con 0x7f9aec072080 2024-04-25T00:56:43.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec072460 con 0x7f9aec072080 2024-04-25T00:56:43.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc004cd0 con 0x7f9aec072080 2024-04-25T00:56:43.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec1e0040 con 0x7f9aec072080 2024-04-25T00:56:43.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9aec068260 con 0x7f9aec072080 2024-04-25T00:56:43.864 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9adc0a5800 con 0x7f9aec072080 2024-04-25T00:56:43.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9adaffd640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9acc0678c0 0x7f9acc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f9adc11f170 con 0x7f9aec072080 2024-04-25T00:56:43.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9ae9d9b640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9acc0678c0 0x7f9acc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9adaffd640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9acc06d780 0x7f9acc06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 --> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9adc008e50 con 0x7f9acc06d780 2024-04-25T00:56:43.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9adc11f600 con 0x7f9aec072080 2024-04-25T00:56:43.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9ae9d9b640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9acc0678c0 0x7f9acc069d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006e90 tx=0x7f9ae0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.865 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9aead9d640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9acc06d780 0x7f9acc06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.854+0000 7f9aead9d640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9acc06d780 0x7f9acc06fba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:43.866 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.858+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 <== osd.7 v2:172.21.15.187:6824/2342593789 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9adc008e50 con 0x7f9acc06d780 2024-04-25T00:56:43.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 --> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9aec13e980 con 0x7f9acc06d780 2024-04-25T00:56:43.873 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9adaffd640 1 -- 172.21.15.112:0/1043384857 <== osd.7 v2:172.21.15.187:6824/2342593789 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f9aec13e980 con 0x7f9acc06d780 2024-04-25T00:56:43.873 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2024-04-25T00:56:43.874 INFO:teuthology.orchestra.run.smithi112.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T00:56:43.874 INFO:teuthology.orchestra.run.smithi112.stdout:} 2024-04-25T00:56:43.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9acc06d780 msgr2=0x7f9acc06fba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.875 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.187:6824/2342593789,v1:172.21.15.187:6825/2342593789] conn(0x7f9acc06d780 0x7f9acc06fba0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9acc0678c0 msgr2=0x7f9acc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f9acc0678c0 0x7f9acc069d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9ae0006e90 tx=0x7f9ae0008040 comp rx=0 tx=0).stop 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec072080 msgr2=0x7f9aec0b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 --2- 172.21.15.112:0/1043384857 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9aec072080 0x7f9aec0b1670 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9adc001980 tx=0x7f9ad4005840 comp rx=0 tx=0).stop 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9aead9d640 1 -- 172.21.15.112:0/1043384857 reap_dead start 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 shutdown_connections 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 >> 172.21.15.112:0/1043384857 conn(0x7f9aec0b8930 msgr2=0x7f9aec0b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 shutdown_connections 2024-04-25T00:56:43.879 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.870+0000 7f9af0ee3640 1 -- 172.21.15.112:0/1043384857 wait complete. 2024-04-25T00:56:43.887 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T00:56:43.994 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.990+0000 7fb78bbb1640 1 Processor -- start 2024-04-25T00:56:43.995 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.990+0000 7fb78bbb1640 1 -- start start 2024-04-25T00:56:43.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78bbb1640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb7840add70 0x7fb784145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:43.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78bbb1640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840aeec0 con 0x7fb7840b3ce0 2024-04-25T00:56:43.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78bbb1640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7840af780 con 0x7fb7840b23f0 2024-04-25T00:56:43.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78bbb1640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fb7840b27d0 con 0x7fb7840add70 2024-04-25T00:56:43.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78a14e640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb7840b3ce0 0x7fb78414fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:36898/0 (socket says 172.21.15.112:36898) 2024-04-25T00:56:43.997 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78a14e640 1 -- 172.21.15.112:0/2727700032 learned_addr learned my addr 172.21.15.112:0/2727700032 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:56:43.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78914c640 1 --2- 172.21.15.112:0/2727700032 >> v2:172.21.15.112:3300/0 conn(0x7fb7840add70 0x7fb784145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:43.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78894b640 1 -- 172.21.15.112:0/2727700032 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771317938 0 0) 0x7fb7840aeec0 con 0x7fb7840b3ce0 2024-04-25T00:56:43.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78894b640 1 -- 172.21.15.112:0/2727700032 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb770003530 con 0x7fb7840b3ce0 2024-04-25T00:56:43.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78914c640 1 -- 172.21.15.112:0/2727700032 >> v1:172.21.15.187:6789/0 conn(0x7fb7840b23f0 legacy=0x7fb784072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78914c640 1 -- 172.21.15.112:0/2727700032 >> v1:172.21.15.112:6789/0 conn(0x7fb7840b3ce0 legacy=0x7fb78414fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:43.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78894b640 1 -- 172.21.15.112:0/2727700032 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4013483788 0 0) 0x7fb770003530 con 0x7fb7840b3ce0 2024-04-25T00:56:43.998 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78914c640 1 -- 172.21.15.112:0/2727700032 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7840afa20 con 0x7fb7840add70 2024-04-25T00:56:44.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78914c640 1 --2- 172.21.15.112:0/2727700032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb784145200 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb7840af950 tx=0x7fb77c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e68b8a4c0ca124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:44.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78894b640 1 -- 172.21.15.112:0/2727700032 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c0b2070 con 0x7fb7840add70 2024-04-25T00:56:44.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78894b640 1 -- 172.21.15.112:0/2727700032 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77c0a56d0 con 0x7fb7840add70 2024-04-25T00:56:44.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.994+0000 7fb78894b640 1 -- 172.21.15.112:0/2727700032 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb774001680 con 0x7fb7840add70 2024-04-25T00:56:44.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- 172.21.15.112:0/2727700032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 msgr2=0x7fb784145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.002 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 --2- 172.21.15.112:0/2727700032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb784145200 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb7840af950 tx=0x7fb77c01c870 comp rx=0 tx=0).stop 2024-04-25T00:56:44.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- 172.21.15.112:0/2727700032 shutdown_connections 2024-04-25T00:56:44.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 --2- 172.21.15.112:0/2727700032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb784145200 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- 172.21.15.112:0/2727700032 >> 172.21.15.112:0/2727700032 conn(0x7fb7840b5980 msgr2=0x7fb7840b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:44.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- 172.21.15.112:0/2727700032 shutdown_connections 2024-04-25T00:56:44.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- 172.21.15.112:0/2727700032 wait complete. 2024-04-25T00:56:44.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 Processor -- start 2024-04-25T00:56:44.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- start start 2024-04-25T00:56:44.003 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb7840711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7840b23f0 0x7fb784134840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7840b3ce0 0x7fb784137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840b52d0 con 0x7fb7840add70 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fb7840b5150 con 0x7fb7840b3ce0 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fb784153a80 con 0x7fb7840b23f0 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb7840711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb7840711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46322/0 (socket says 172.21.15.112:46322) 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 -- 172.21.15.112:0/3883412630 learned_addr learned my addr 172.21.15.112:0/3883412630 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:44.004 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78a14e640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7840b3ce0 0x7fb784137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78914c640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7840b23f0 0x7fb784134840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 -- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7840b23f0 msgr2=0x7fb784134840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7840b23f0 0x7fb784134840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 -- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7840b3ce0 msgr2=0x7fb784137d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7840b3ce0 0x7fb784137d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 -- 172.21.15.112:0/3883412630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb784140ef0 con 0x7fb7840add70 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78a14e640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7840b3ce0 0x7fb784137d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78994d640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb7840711a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c500 tx=0x7fb78000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:44.005 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb77a7fc640 1 -- 172.21.15.112:0/3883412630 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780016020 con 0x7fb7840add70 2024-04-25T00:56:44.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb77a7fc640 1 -- 172.21.15.112:0/3883412630 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb780005220 con 0x7fb7840add70 2024-04-25T00:56:44.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7840709a0 con 0x7fb7840add70 2024-04-25T00:56:44.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb77a7fc640 1 -- 172.21.15.112:0/3883412630 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780007030 con 0x7fb7840add70 2024-04-25T00:56:44.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb784138a30 con 0x7fb7840add70 2024-04-25T00:56:44.006 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78404f580 con 0x7fb7840add70 2024-04-25T00:56:44.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:43.998+0000 7fb77a7fc640 1 -- 172.21.15.112:0/3883412630 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7800071d0 con 0x7fb7840add70 2024-04-25T00:56:44.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.002+0000 7fb77a7fc640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb7680678f0 0x7fb768069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.002+0000 7fb77a7fc640 1 -- 172.21.15.112:0/3883412630 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fb780084e00 con 0x7fb7840add70 2024-04-25T00:56:44.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.002+0000 7fb78914c640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb7680678f0 0x7fb768069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.002+0000 7fb78914c640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb7680678f0 0x7fb768069db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0af000 tx=0x7fb77c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:44.008 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.006+0000 7fb77a7fc640 1 -- 172.21.15.112:0/3883412630 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb780055c50 con 0x7fb7840add70 2024-04-25T00:56:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fb7840b8200 con 0x7fb7680678f0 2024-04-25T00:56:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb77a7fc640 1 -- 172.21.15.112:0/3883412630 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb7840b8200 con 0x7fb7680678f0 2024-04-25T00:56:44.161 INFO:teuthology.orchestra.run.smithi112.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T00:56:44.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb7680678f0 msgr2=0x7fb768069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb7680678f0 0x7fb768069db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb77c0af000 tx=0x7fb77c0a7070 comp rx=0 tx=0).stop 2024-04-25T00:56:44.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 msgr2=0x7fb7840711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb7840711a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c500 tx=0x7fb78000c9c0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 shutdown_connections 2024-04-25T00:56:44.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fb7680678f0 0x7fb768069db0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fb7840b3ce0 0x7fb784137d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.163 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fb7840b23f0 0x7fb784134840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 --2- 172.21.15.112:0/3883412630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7840add70 0x7fb7840711a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 >> 172.21.15.112:0/3883412630 conn(0x7fb7840b5980 msgr2=0x7fb78406c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:44.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 shutdown_connections 2024-04-25T00:56:44.164 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.158+0000 7fb78bbb1640 1 -- 172.21.15.112:0/3883412630 wait complete. 2024-04-25T00:56:44.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:56:44.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.282+0000 7fdba5694640 1 Processor -- start 2024-04-25T00:56:44.287 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.282+0000 7fdba5694640 1 -- start start 2024-04-25T00:56:44.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.282+0000 7fdba5694640 1 --2- >> v2:172.21.15.187:3300/0 conn(0x7fdba006c520 0x7fdba00727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.282+0000 7fdba5694640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba00b47d0 0x7fdba00b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.282+0000 7fdba5694640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdba0141110 0x7fdba01415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.282+0000 7fdba5694640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7fdba00b5300 con 0x7fdba0141110 2024-04-25T00:56:44.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.282+0000 7fdba5694640 1 -- --> v2:172.21.15.187:3300/0 -- mon_getmap magic: 0 -- 0x7fdba006c900 con 0x7fdba006c520 2024-04-25T00:56:44.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.282+0000 7fdba5694640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdba00ae670 con 0x7fdba00b47d0 2024-04-25T00:56:44.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdba0141110 0x7fdba01415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.288 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdba0141110 0x7fdba01415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46330/0 (socket says 172.21.15.112:46330) 2024-04-25T00:56:44.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 -- 172.21.15.112:0/2238017899 learned_addr learned my addr 172.21.15.112:0/2238017899 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:44.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9effd640 1 --2- 172.21.15.112:0/2238017899 >> v2:172.21.15.187:3300/0 conn(0x7fdba006c520 0x7fdba00727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9e7fc640 1 --2- 172.21.15.112:0/2238017899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba00b47d0 0x7fdba00b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 -- 172.21.15.112:0/2238017899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba00b47d0 msgr2=0x7fdba00b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:44.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 --2- 172.21.15.112:0/2238017899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba00b47d0 0x7fdba00b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 -- 172.21.15.112:0/2238017899 >> v2:172.21.15.187:3300/0 conn(0x7fdba006c520 msgr2=0x7fdba00727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:56:44.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 --2- 172.21.15.112:0/2238017899 >> v2:172.21.15.187:3300/0 conn(0x7fdba006c520 0x7fdba00727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.289 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 -- 172.21.15.112:0/2238017899 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba00ae950 con 0x7fdba0141110 2024-04-25T00:56:44.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9f7fe640 1 --2- 172.21.15.112:0/2238017899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba0141110 0x7fdba01415a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdba00b9bc0 tx=0x7fdb9801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850edc56910e4888 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:44.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9dffb640 1 -- 172.21.15.112:0/2238017899 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980b0070 con 0x7fdba0141110 2024-04-25T00:56:44.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9dffb640 1 -- 172.21.15.112:0/2238017899 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb980ab600 con 0x7fdba0141110 2024-04-25T00:56:44.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdb9dffb640 1 -- 172.21.15.112:0/2238017899 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980ab9d0 con 0x7fdba0141110 2024-04-25T00:56:44.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 -- 172.21.15.112:0/2238017899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba0141110 msgr2=0x7fdba01415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.291 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 --2- 172.21.15.112:0/2238017899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba0141110 0x7fdba01415a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdba00b9bc0 tx=0x7fdb9801c490 comp rx=0 tx=0).stop 2024-04-25T00:56:44.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 -- 172.21.15.112:0/2238017899 shutdown_connections 2024-04-25T00:56:44.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 --2- 172.21.15.112:0/2238017899 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba0141110 0x7fdba01415a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 --2- 172.21.15.112:0/2238017899 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba00b47d0 0x7fdba00b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 --2- 172.21.15.112:0/2238017899 >> v2:172.21.15.187:3300/0 conn(0x7fdba006c520 0x7fdba00727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 -- 172.21.15.112:0/2238017899 >> 172.21.15.112:0/2238017899 conn(0x7fdba00b5980 msgr2=0x7fdba00b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:44.292 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 -- 172.21.15.112:0/2238017899 shutdown_connections 2024-04-25T00:56:44.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.286+0000 7fdba5694640 1 -- 172.21.15.112:0/2238017899 wait complete. 2024-04-25T00:56:44.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 Processor -- start 2024-04-25T00:56:44.293 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 -- start start 2024-04-25T00:56:44.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba006c520 0x7fdba01e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.295 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00b47d0 0x7fdba0141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba0141110 0x7fdba0142320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7fdba014b370 con 0x7fdba006c520 2024-04-25T00:56:44.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7fdba014b1f0 con 0x7fdba00b47d0 2024-04-25T00:56:44.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7fdba014b4f0 con 0x7fdba0141110 2024-04-25T00:56:44.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba006c520 0x7fdba01e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba006c520 0x7fdba01e34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:46338/0 (socket says 172.21.15.112:46338) 2024-04-25T00:56:44.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 -- 172.21.15.112:0/2878675606 learned_addr learned my addr 172.21.15.112:0/2878675606 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:56:44.296 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9e7fc640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00b47d0 0x7fdba0141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9f7fe640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba0141110 0x7fdba0142320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 -- 172.21.15.112:0/2878675606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba0141110 msgr2=0x7fdba0142320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba0141110 0x7fdba0142320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 -- 172.21.15.112:0/2878675606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00b47d0 msgr2=0x7fdba0141de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00b47d0 0x7fdba0141de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 -- 172.21.15.112:0/2878675606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba00b8f00 con 0x7fdba006c520 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9effd640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba006c520 0x7fdba01e34e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800d800 tx=0x7fdb8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb977fe640 1 -- 172.21.15.112:0/2878675606 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8800b6c0 con 0x7fdba006c520 2024-04-25T00:56:44.297 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb977fe640 1 -- 172.21.15.112:0/2878675606 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb88019070 con 0x7fdba006c520 2024-04-25T00:56:44.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba00b91e0 con 0x7fdba006c520 2024-04-25T00:56:44.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb977fe640 1 -- 172.21.15.112:0/2878675606 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8800ba60 con 0x7fdba006c520 2024-04-25T00:56:44.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba00b97a0 con 0x7fdba006c520 2024-04-25T00:56:44.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba004f580 con 0x7fdba006c520 2024-04-25T00:56:44.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb977fe640 1 -- 172.21.15.112:0/2878675606 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdb8800bc00 con 0x7fdba006c520 2024-04-25T00:56:44.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb977fe640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdb800678f0 0x7fdb80069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:56:44.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb9e7fc640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdb800678f0 0x7fdb80069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:56:44.298 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.290+0000 7fdb977fe640 1 -- 172.21.15.112:0/2878675606 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7fdb88085da0 con 0x7fdba006c520 2024-04-25T00:56:44.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.294+0000 7fdb9e7fc640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdb800678f0 0x7fdb80069db0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdba0071a90 tx=0x7fdb9000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:56:44.299 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.294+0000 7fdb977fe640 1 -- 172.21.15.112:0/2878675606 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdb88056c70 con 0x7fdba006c520 2024-04-25T00:56:44.444 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.442+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdba01e4130 con 0x7fdb800678f0 2024-04-25T00:56:44.446 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.442+0000 7fdb977fe640 1 -- 172.21.15.112:0/2878675606 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7fdba01e4130 con 0x7fdb800678f0 2024-04-25T00:56:44.446 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:56:44.447 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:56:44.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdb800678f0 msgr2=0x7fdb80069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdb800678f0 0x7fdb80069db0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdba0071a90 tx=0x7fdb9000b040 comp rx=0 tx=0).stop 2024-04-25T00:56:44.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba006c520 msgr2=0x7fdba01e34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:56:44.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba006c520 0x7fdba01e34e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800d800 tx=0x7fdb8800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 shutdown_connections 2024-04-25T00:56:44.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7fdba0141110 0x7fdba0142320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7fdb800678f0 0x7fdb80069db0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.449 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7fdba00b47d0 0x7fdba0141de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 --2- 172.21.15.112:0/2878675606 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdba006c520 0x7fdba01e34e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:56:44.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 >> 172.21.15.112:0/2878675606 conn(0x7fdba00b5980 msgr2=0x7fdba00aee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:56:44.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 shutdown_connections 2024-04-25T00:56:44.450 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:56:44.446+0000 7fdba5694640 1 -- 172.21.15.112:0/2878675606 wait complete. 2024-04-25T00:56:44.464 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-04-25T00:56:43.531569+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":224372,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750348,"statfs":{"total":773094113280,"available":772864356352,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001852"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":194,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T00:56:37.468680+0000","last_change":"2024-04-25T00:55:28.779226+0000","last_active":"2024-04-25T00:56:37.468680+0000","last_peered":"2024-04-25T00:56:37.468680+0000","last_clean":"2024-04-25T00:56:37.468680+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:56:37.468680+0000","last_undegraded":"2024-04-25T00:56:37.468680+0000","last_fullsized":"2024-04-25T00:56:37.468680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:55:27.652287+0000","last_clean_scrub_stamp":"2024-04-25T00:55:27.652287+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:37:16.281055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:56:44.466 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T00:55:27.652287+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=55, tm_sec=27, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=0, tm_min=56, tm_sec=39, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T00:56:44.466 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T00:57:04.468 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:57:04.598 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.593+0000 7f52ef68a640 1 Processor -- start 2024-04-25T00:57:04.599 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.593+0000 7f52ef68a640 1 -- start start 2024-04-25T00:57:04.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ef68a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e80add70 0x7f52e80aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ef68a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e81485e0 con 0x7f52e806ca10 2024-04-25T00:57:04.601 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ef68a640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e80af760 con 0x7f52e806c630 2024-04-25T00:57:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ef68a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52e8148860 con 0x7f52e80add70 2024-04-25T00:57:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ecc25640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f52e806ca10 0x7f52e8147ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:53996/0 (socket says 172.21.15.112:53996) 2024-04-25T00:57:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ecc25640 1 -- 172.21.15.112:0/1340674548 learned_addr learned my addr 172.21.15.112:0/1340674548 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:57:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52edc27640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e80add70 0x7f52e80aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4095509049 0 0) 0x7f52e81485e0 con 0x7f52e806ca10 2024-04-25T00:57:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52d4003530 con 0x7f52e806ca10 2024-04-25T00:57:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2558943279 0 0) 0x7f52d4003530 con 0x7f52e806ca10 2024-04-25T00:57:04.602 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e80add70 msgr2=0x7f52e80aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:57:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e80add70 0x7f52e80aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 >> v1:172.21.15.187:6789/0 conn(0x7f52e806c630 legacy=0x7f52e81477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80afa00 con 0x7f52e806ca10 2024-04-25T00:57:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f52e4001680 con 0x7f52e806ca10 2024-04-25T00:57:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 >> v1:172.21.15.112:6789/0 conn(0x7f52e806ca10 legacy=0x7f52e8147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d40039f0 0x7f52d4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e80add70 0x7f52d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.603 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d4004d50 0x7f52d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52d400bb60 con 0x7f52d4004d50 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52d400bd10 con 0x7f52d40039f0 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52d400bf10 con 0x7f52e80add70 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52edc27640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d4004d50 0x7f52d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ecc25640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e80add70 0x7f52d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ed426640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d40039f0 0x7f52d4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52edc27640 1 -- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e80add70 msgr2=0x7f52d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52edc27640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e80add70 0x7f52d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.604 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52edc27640 1 -- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d40039f0 msgr2=0x7f52d4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:57:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52edc27640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d40039f0 0x7f52d4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52edc27640 1 -- 172.21.15.112:0/1340674548 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d4003530 con 0x7f52d4004d50 2024-04-25T00:57:04.605 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.597+0000 7f52ed426640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d40039f0 0x7f52d4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:57:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d4004d50 0x7f52d40051d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009f90 tx=0x7f52e001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625bad84d84ca2ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e00b2070 con 0x7f52d4004d50 2024-04-25T00:57:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e00a5a30 con 0x7f52d4004d50 2024-04-25T00:57:04.609 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52dffff640 1 -- 172.21.15.112:0/1340674548 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0009c10 con 0x7f52d4004d50 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- 172.21.15.112:0/1340674548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d4004d50 msgr2=0x7f52d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 --2- 172.21.15.112:0/1340674548 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52d4004d50 0x7f52d40051d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009f90 tx=0x7f52e001cc40 comp rx=0 tx=0).stop 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 -- 172.21.15.112:0/1340674548 reap_dead start 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- 172.21.15.112:0/1340674548 shutdown_connections 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- 172.21.15.112:0/1340674548 >> 172.21.15.112:0/1340674548 conn(0x7f52e80b5980 msgr2=0x7f52e80b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- 172.21.15.112:0/1340674548 shutdown_connections 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- 172.21.15.112:0/1340674548 wait complete. 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 Processor -- start 2024-04-25T00:57:04.610 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- start start 2024-04-25T00:57:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d4004d50 0x7f52e8143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e809bcf0 0x7f52e81de730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e81dec70 0x7f52e81e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f52e8150870 con 0x7f52e81dec70 2024-04-25T00:57:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80b1060 con 0x7f52d4004d50 2024-04-25T00:57:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f52e81509f0 con 0x7f52e809bcf0 2024-04-25T00:57:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e81dec70 0x7f52e81e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.611 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e81dec70 0x7f52e81e16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43772/0 (socket says 172.21.15.112:43772) 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 -- 172.21.15.112:0/2768127682 learned_addr learned my addr 172.21.15.112:0/2768127682 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ed426640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d4004d50 0x7f52e8143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ecc25640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e809bcf0 0x7f52e81de730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 -- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e809bcf0 msgr2=0x7f52e81de730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e809bcf0 0x7f52e81de730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 -- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d4004d50 msgr2=0x7f52e8143300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d4004d50 0x7f52e8143300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 -- 172.21.15.112:0/2768127682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e81de280 con 0x7f52e81dec70 2024-04-25T00:57:04.612 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ed426640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d4004d50 0x7f52e8143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52edc27640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e81dec70 0x7f52e81e16e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f52e00b0040 tx=0x7f52e0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ddffb640 1 -- 172.21.15.112:0/2768127682 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e00bc070 con 0x7f52e81dec70 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ddffb640 1 -- 172.21.15.112:0/2768127682 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e00ad440 con 0x7f52e81dec70 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e81df100 con 0x7f52e81dec70 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ddffb640 1 -- 172.21.15.112:0/2768127682 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e00b2040 con 0x7f52e81dec70 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.601+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e809b020 con 0x7f52e81dec70 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.605+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e804f580 con 0x7f52e81dec70 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.605+0000 7f52ddffb640 1 -- 172.21.15.112:0/2768127682 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f52e00bf070 con 0x7f52e81dec70 2024-04-25T00:57:04.613 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.605+0000 7f52ddffb640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52cc0678f0 0x7f52cc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.605+0000 7f52ddffb640 1 -- 172.21.15.112:0/2768127682 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f52e011f070 con 0x7f52e81dec70 2024-04-25T00:57:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.605+0000 7f52ed426640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52cc0678f0 0x7f52cc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.605+0000 7f52ed426640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52cc0678f0 0x7f52cc069db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f52d80017e0 tx=0x7f52d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:04.614 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.609+0000 7f52ddffb640 1 -- 172.21.15.112:0/2768127682 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f52e0016950 con 0x7f52e81dec70 2024-04-25T00:57:04.771 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 --> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52e80b1e20 con 0x7f52cc0678f0 2024-04-25T00:57:04.772 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ddffb640 1 -- 172.21.15.112:0/2768127682 <== mgr.4104 v2:172.21.15.112:6832/80465356 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13354 (secure 0 0 0) 0x7f52e80b1e20 con 0x7f52cc0678f0 2024-04-25T00:57:04.772 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:57:04.772 INFO:teuthology.orchestra.run.smithi112.stderr:dumped all 2024-04-25T00:57:04.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52cc0678f0 msgr2=0x7f52cc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52cc0678f0 0x7f52cc069db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f52d80017e0 tx=0x7f52d8009040 comp rx=0 tx=0).stop 2024-04-25T00:57:04.774 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e81dec70 msgr2=0x7f52e81e16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e81dec70 0x7f52e81e16e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f52e00b0040 tx=0x7f52e0009750 comp rx=0 tx=0).stop 2024-04-25T00:57:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 shutdown_connections 2024-04-25T00:57:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52e81dec70 0x7f52e81e16e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f52e809bcf0 0x7f52e81de730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f52cc0678f0 0x7f52cc069db0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.775 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 --2- 172.21.15.112:0/2768127682 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f52d4004d50 0x7f52e8143300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 >> 172.21.15.112:0/2768127682 conn(0x7f52e80b5980 msgr2=0x7f52e80ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:57:04.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 shutdown_connections 2024-04-25T00:57:04.776 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.769+0000 7f52ef68a640 1 -- 172.21.15.112:0/2768127682 wait complete. 2024-04-25T00:57:04.791 INFO:teuthology.orchestra.run.smithi112.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-25T00:57:03.536631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":224364,"kb_used_data":4040,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750356,"statfs":{"total":773094113280,"available":772864364544,"internally_reserved":0,"allocated":4136960,"data_stored":2314024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003070"},"pg_stats":[{"pgid":"1.0","version":"32'119","reported_seq":205,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T00:56:44.947958+0000","last_change":"2024-04-25T00:56:44.948034+0000","last_active":"2024-04-25T00:56:44.947958+0000","last_peered":"2024-04-25T00:56:44.947958+0000","last_clean":"2024-04-25T00:56:44.947958+0000","last_became_active":"2024-04-25T00:55:28.777336+0000","last_became_peered":"2024-04-25T00:55:28.777336+0000","last_unstale":"2024-04-25T00:56:44.947958+0000","last_undegraded":"2024-04-25T00:56:44.947958+0000","last_fullsized":"2024-04-25T00:56:44.947958+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'119","last_scrub_stamp":"2024-04-25T00:56:44.947773+0000","last_deep_scrub":"32'119","last_deep_scrub_stamp":"2024-04-25T00:56:44.947773+0000","last_clean_scrub_stamp":"2024-04-25T00:56:44.947773+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:57:44.827955+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27380,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":368640,"data_stored":141661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":962560,"data_stored":732029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:57:04.792 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T00:57:04.915 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f61152640 1 Processor -- start 2024-04-25T00:57:04.916 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f61152640 1 -- start start 2024-04-25T00:57:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f61152640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f5c0b9030 0x7f7f5c09acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f61152640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f5c0b9410 con 0x7f7f5c14fcd0 2024-04-25T00:57:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f61152640 1 -- --> v1:172.21.15.187:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f5c0baa90 con 0x7f7f5c09b800 2024-04-25T00:57:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f61152640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5c0b9990 con 0x7f7f5c0b9030 2024-04-25T00:57:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5b59e640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f7f5c14fcd0 0x7f7f5c1520c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54012/0 (socket says 172.21.15.112:54012) 2024-04-25T00:57:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5b59e640 1 -- 172.21.15.112:0/4177422958 learned_addr learned my addr 172.21.15.112:0/4177422958 (peer_addr_for_me v1:172.21.15.112:0/0) 2024-04-25T00:57:04.919 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5ad9d640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f5c0b9030 0x7f7f5c09acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1499516958 0 0) 0x7f7f5c0b9410 con 0x7f7f5c14fcd0 2024-04-25T00:57:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f44003530 con 0x7f7f5c14fcd0 2024-04-25T00:57:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 <== mon.0 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 434125427 0 0) 0x7f7f44003530 con 0x7f7f5c14fcd0 2024-04-25T00:57:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f5c0b9030 msgr2=0x7f7f5c09acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:57:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f5c0b9030 0x7f7f5c09acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 >> v1:172.21.15.187:6789/0 conn(0x7f7f5c09b800 legacy=0x7f7f5c141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.920 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c0bad70 con 0x7f7f5c14fcd0 2024-04-25T00:57:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 <== mon.0 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 300172554 0 0) 0x7f7f54001a00 con 0x7f7f5c14fcd0 2024-04-25T00:57:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 >> v1:172.21.15.112:6789/0 conn(0x7f7f5c14fcd0 legacy=0x7f7f5c1520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f5c0b9030 0x7f7f440039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44003f70 0x7f7f44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f44004d50 0x7f7f440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c0b9990 con 0x7f7f44003f70 2024-04-25T00:57:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4400bb60 con 0x7f7f44004d50 2024-04-25T00:57:04.921 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4400bd60 con 0x7f7f5c0b9030 2024-04-25T00:57:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5a59c640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44003f70 0x7f7f44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5ad9d640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f5c0b9030 0x7f7f440039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5b59e640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f44004d50 0x7f7f440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5a59c640 1 -- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f5c0b9030 msgr2=0x7f7f440039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:57:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5a59c640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f5c0b9030 0x7f7f440039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5a59c640 1 -- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f44004d50 msgr2=0x7f7f440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:57:04.922 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5a59c640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f44004d50 0x7f7f440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.923 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.913+0000 7f7f5a59c640 1 -- 172.21.15.112:0/4177422958 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f44003530 con 0x7f7f44003f70 2024-04-25T00:57:04.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f5a59c640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44003f70 0x7f7f44004810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7f50001750 tx=0x7f7f5001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5e6e0c599e40b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:04.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500013a0 con 0x7f7f44003f70 2024-04-25T00:57:04.927 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f500052f0 con 0x7f7f44003f70 2024-04-25T00:57:04.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f59d9b640 1 -- 172.21.15.112:0/4177422958 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50003840 con 0x7f7f44003f70 2024-04-25T00:57:04.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- 172.21.15.112:0/4177422958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44003f70 msgr2=0x7f7f44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 --2- 172.21.15.112:0/4177422958 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f44003f70 0x7f7f44004810 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7f50001750 tx=0x7f7f5001c3e0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f5b59e640 1 -- 172.21.15.112:0/4177422958 reap_dead start 2024-04-25T00:57:04.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- 172.21.15.112:0/4177422958 shutdown_connections 2024-04-25T00:57:04.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- 172.21.15.112:0/4177422958 >> 172.21.15.112:0/4177422958 conn(0x7f7f5c09eaf0 msgr2=0x7f7f5c09ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:57:04.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- 172.21.15.112:0/4177422958 shutdown_connections 2024-04-25T00:57:04.928 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- 172.21.15.112:0/4177422958 wait complete. 2024-04-25T00:57:04.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 Processor -- start 2024-04-25T00:57:04.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- start start 2024-04-25T00:57:04.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f44003f70 0x7f7f5c09e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5c0b35f0 0x7f7f5c139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0b2380 0x7f7f5c1e3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.929 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c141be0 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c141a60 con 0x7f7f5c0b2380 2024-04-25T00:57:04.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.917+0000 7f7f61152640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5c09c140 con 0x7f7f44003f70 2024-04-25T00:57:04.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5c0b35f0 0x7f7f5c139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5c0b35f0 0x7f7f5c139b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43776/0 (socket says 172.21.15.112:43776) 2024-04-25T00:57:04.930 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 -- 172.21.15.112:0/1793753367 learned_addr learned my addr 172.21.15.112:0/1793753367 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:57:04.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5b59e640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0b2380 0x7f7f5c1e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5ad9d640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f44003f70 0x7f7f5c09e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 -- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f44003f70 msgr2=0x7f7f5c09e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f44003f70 0x7f7f5c09e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 -- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0b2380 msgr2=0x7f7f5c1e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:04.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0b2380 0x7f7f5c1e3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:04.931 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 -- 172.21.15.112:0/1793753367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c0b2b90 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5b59e640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0b2380 0x7f7f5c1e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:57:04.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5a59c640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5c0b35f0 0x7f7f5c139b40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005040 tx=0x7f7f44005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:04.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f437fe640 1 -- 172.21.15.112:0/1793753367 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500b9020 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f437fe640 1 -- 172.21.15.112:0/1793753367 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f500054a0 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c13b5c0 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.932 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f437fe640 1 -- 172.21.15.112:0/1793753367 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f50007a40 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c1e3760 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5c04f5f0 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f437fe640 1 -- 172.21.15.112:0/1793753367 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7f500bc020 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f437fe640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7f480678f0 0x7f7f48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:04.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f437fe640 1 -- 172.21.15.112:0/1793753367 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f7f5011fe10 con 0x7f7f5c0b35f0 2024-04-25T00:57:04.933 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5ad9d640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7f480678f0 0x7f7f48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:04.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.921+0000 7f7f5ad9d640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7f480678f0 0x7f7f48069db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009a10 tx=0x7f7f4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:04.934 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:04.925+0000 7f7f437fe640 1 -- 172.21.15.112:0/1793753367 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7f500c3030 con 0x7f7f5c0b35f0 2024-04-25T00:57:05.085 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.081+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f7f5c09ccd0 con 0x7f7f5c0b35f0 2024-04-25T00:57:05.090 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.085+0000 7f7f437fe640 1 -- 172.21.15.112:0/1793753367 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f7f500160f0 con 0x7f7f5c0b35f0 2024-04-25T00:57:05.091 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f437fe640 1 -- 172.21.15.112:0/1793753367 <== mon.0 v2:172.21.15.112:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7f500f0dc0 con 0x7f7f5c0b35f0 2024-04-25T00:57:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7f480678f0 msgr2=0x7f7f48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7f480678f0 0x7f7f48069db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009a10 tx=0x7f7f4c008040 comp rx=0 tx=0).stop 2024-04-25T00:57:05.093 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5c0b35f0 msgr2=0x7f7f5c139b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5c0b35f0 0x7f7f5c139b40 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7f50005040 tx=0x7f7f44005840 comp rx=0 tx=0).stop 2024-04-25T00:57:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 shutdown_connections 2024-04-25T00:57:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f7f480678f0 0x7f7f48069db0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f7f5c0b2380 0x7f7f5c1e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7f5c0b35f0 0x7f7f5c139b40 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 --2- 172.21.15.112:0/1793753367 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f7f44003f70 0x7f7f5c09e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 >> 172.21.15.112:0/1793753367 conn(0x7f7f5c09eaf0 msgr2=0x7f7f5c072ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:57:05.094 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 shutdown_connections 2024-04-25T00:57:05.095 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.089+0000 7f7f61152640 1 -- 172.21.15.112:0/1793753367 wait complete. 2024-04-25T00:57:05.109 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:57:05.110 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:57:05.227 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.221+0000 7f041fb98640 1 Processor -- start 2024-04-25T00:57:05.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.221+0000 7f041fb98640 1 -- start start 2024-04-25T00:57:05.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 0x7f04180ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:05.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04180aee00 0x7f04180b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:05.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04180b52f0 con 0x7f04180ae9a0 2024-04-25T00:57:05.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 -- 0x7f04180b0800 con 0x7f04180aee00 2024-04-25T00:57:05.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f0418143f50 con 0x7f04180adfd0 2024-04-25T00:57:05.228 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041e135640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04180aee00 0x7f04180b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041e135640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f04180aee00 0x7f04180b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43784/0 (socket says 172.21.15.112:43784) 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041e135640 1 -- 172.21.15.112:0/2956316130 learned_addr learned my addr 172.21.15.112:0/2956316130 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041d133640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f04180ae9a0 0x7f04180b2240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:54022/0 (socket says 172.21.15.112:54022) 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041e135640 1 -- 172.21.15.112:0/2956316130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 msgr2=0x7f04180ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041d934640 1 --2- 172.21.15.112:0/2956316130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 0x7f04180ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041e135640 1 --2- 172.21.15.112:0/2956316130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 0x7f04180ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041e135640 1 -- 172.21.15.112:0/2956316130 >> v1:172.21.15.112:6789/0 conn(0x7f04180ae9a0 legacy=0x7f04180b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041c932640 1 -- 172.21.15.112:0/2956316130 <== mon.0 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513158657 0 0) 0x7f04180b52f0 con 0x7f04180ae9a0 2024-04-25T00:57:05.229 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041e135640 1 -- 172.21.15.112:0/2956316130 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418144230 con 0x7f04180aee00 2024-04-25T00:57:05.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041e135640 1 --2- 172.21.15.112:0/2956316130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180aee00 0x7f04180b2950 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f041000af60 tx=0x7f041001daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12e0ef8a8fc2fda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:05.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041c932640 1 -- 172.21.15.112:0/2956316130 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041000a2d0 con 0x7f04180aee00 2024-04-25T00:57:05.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041c932640 1 -- 172.21.15.112:0/2956316130 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f04100a68a0 con 0x7f04180aee00 2024-04-25T00:57:05.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041c932640 1 -- 172.21.15.112:0/2956316130 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100a6c90 con 0x7f04180aee00 2024-04-25T00:57:05.231 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 -- 172.21.15.112:0/2956316130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180aee00 msgr2=0x7f04180b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 --2- 172.21.15.112:0/2956316130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180aee00 0x7f04180b2950 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f041000af60 tx=0x7f041001daa0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 -- 172.21.15.112:0/2956316130 shutdown_connections 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 --2- 172.21.15.112:0/2956316130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180aee00 0x7f04180b2950 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 --2- 172.21.15.112:0/2956316130 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 0x7f04180ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 -- 172.21.15.112:0/2956316130 >> 172.21.15.112:0/2956316130 conn(0x7f04180b5980 msgr2=0x7f04180b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 -- 172.21.15.112:0/2956316130 shutdown_connections 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.225+0000 7f041fb98640 1 -- 172.21.15.112:0/2956316130 wait complete. 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 Processor -- start 2024-04-25T00:57:05.232 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 -- start start 2024-04-25T00:57:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 --2- >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 0x7f041813ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180ae9a0 0x7f041813b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 --2- >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04180aee00 0x7f04181e3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 -- 0x7f0418150360 con 0x7f04180ae9a0 2024-04-25T00:57:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 -- --> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] -- mon_getmap magic: 0 -- 0x7f04181501e0 con 0x7f04180aee00 2024-04-25T00:57:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 -- --> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] -- mon_getmap magic: 0 -- 0x7f04181504e0 con 0x7f04180adfd0 2024-04-25T00:57:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180ae9a0 0x7f041813b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:05.236 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180ae9a0 0x7f041813b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:43800/0 (socket says 172.21.15.112:43800) 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 -- 172.21.15.112:0/1256157475 learned_addr learned my addr 172.21.15.112:0/1256157475 (peer_addr_for_me v2:172.21.15.112:0/0) 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 -- 172.21.15.112:0/1256157475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 msgr2=0x7f041813ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 0x7f041813ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 -- 172.21.15.112:0/1256157475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04180aee00 msgr2=0x7f04181e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04180aee00 0x7f04181e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 -- 172.21.15.112:0/1256157475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0418141870 con 0x7f04180ae9a0 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041d133640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180ae9a0 0x7f041813b2a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f041400c4e0 tx=0x7f041400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f040e7fc640 1 -- 172.21.15.112:0/1256157475 <== mon.0 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414018070 con 0x7f04180ae9a0 2024-04-25T00:57:05.237 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f040e7fc640 1 -- 172.21.15.112:0/1256157475 <== mon.0 v2:172.21.15.112:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f04140074c0 con 0x7f04180ae9a0 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041809bd80 con 0x7f04180ae9a0 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f040e7fc640 1 -- 172.21.15.112:0/1256157475 <== mon.0 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414004a50 con 0x7f04180ae9a0 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0418142300 con 0x7f04180ae9a0 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041804f580 con 0x7f04180ae9a0 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.229+0000 7f040e7fc640 1 -- 172.21.15.112:0/1256157475 <== mon.0 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f041401b070 con 0x7f04180ae9a0 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.233+0000 7f040e7fc640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03fc0678f0 0x7f03fc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.233+0000 7f041d934640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03fc0678f0 0x7f03fc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.233+0000 7f040e7fc640 1 -- 172.21.15.112:0/1256157475 <== mon.0 v2:172.21.15.112:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4943+0+0 (secure 0 0 0) 0x7f0414085180 con 0x7f04180ae9a0 2024-04-25T00:57:05.238 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.233+0000 7f041d934640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03fc0678f0 0x7f03fc069db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f04180b84b0 tx=0x7f0408008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:57:05.239 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.233+0000 7f040e7fc640 1 -- 172.21.15.112:0/1256157475 <== mon.0 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0414022020 con 0x7f04180ae9a0 2024-04-25T00:57:05.433 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.429+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04180702a0 con 0x7f04180ae9a0 2024-04-25T00:57:05.434 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.429+0000 7f040e7fc640 1 -- 172.21.15.112:0/1256157475 <== mon.0 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+2225 (secure 0 0 0) 0x7f04140560f0 con 0x7f04180ae9a0 2024-04-25T00:57:05.434 INFO:teuthology.orchestra.run.smithi112.stdout: 2024-04-25T00:57:05.434 INFO:teuthology.orchestra.run.smithi112.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4538,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.187:6833/3898281312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6832","nonce":3898281312},{"type":"v1","addr":"172.21.15.187:6833","nonce":3898281312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6835/2010724842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":2010724842},{"type":"v1","addr":"172.21.15.112:6835","nonce":2010724842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-25T00:57:05.435 INFO:teuthology.orchestra.run.smithi112.stderr:dumped fsmap epoch 18 2024-04-25T00:57:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03fc0678f0 msgr2=0x7f03fc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03fc0678f0 0x7f03fc069db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f04180b84b0 tx=0x7f0408008040 comp rx=0 tx=0).stop 2024-04-25T00:57:05.436 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180ae9a0 msgr2=0x7f041813b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:57:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180ae9a0 0x7f041813b2a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f041400c4e0 tx=0x7f041400c9a0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 shutdown_connections 2024-04-25T00:57:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:6832/80465356,v1:172.21.15.112:6833/80465356] conn(0x7f03fc0678f0 0x7f03fc069db0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] conn(0x7f04180aee00 0x7f04181e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f04180ae9a0 0x7f041813b2a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 --2- 172.21.15.112:0/1256157475 >> [v2:172.21.15.187:3301/0,v1:172.21.15.187:6790/0] conn(0x7f04180adfd0 0x7f041813ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:57:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 >> 172.21.15.112:0/1256157475 conn(0x7f04180b5980 msgr2=0x7f04180b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:57:05.437 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 shutdown_connections 2024-04-25T00:57:05.438 INFO:teuthology.orchestra.run.smithi112.stderr:2024-04-25T00:57:05.433+0000 7f041fb98640 1 -- 172.21.15.112:0/1256157475 wait complete. 2024-04-25T00:57:05.453 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T00:57:05.454 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T00:57:05.454 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T00:57:05.454 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:05.463 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T00:57:05.464 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T00:57:05.464 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:05.482 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T00:57:05.482 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T00:57:05.482 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T00:57:05.483 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:05.579 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T00:57:05.579 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T00:57:05.579 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:05.661 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T00:57:05.662 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T00:57:05.662 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:05.744 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T00:57:05.744 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T00:57:05.744 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:05.819 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T00:57:05.820 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T00:57:05.820 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:05.889 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T00:57:05.889 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T00:57:05.890 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:05.986 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T00:57:05.987 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T00:57:05.987 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:06.086 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T00:57:06.087 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T00:57:06.087 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:06.171 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T00:57:06.171 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T00:57:06.172 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T00:57:06.172 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:06.200 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T00:57:06.200 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T00:57:06.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:06.228 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T00:57:06.228 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T00:57:06.228 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T00:57:06.229 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:06.261 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T00:57:06.262 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T00:57:06.262 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:06.285 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T00:57:06.285 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T00:57:06.285 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:57:06.346 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T00:57:06.346 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T00:57:06.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T00:57:06.365 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:57:06.365 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T00:57:06.636 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:57:06.636 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T00:57:06.856 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:57:06.857 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T00:57:07.038 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:57:07.039 DEBUG:teuthology.orchestra.run.smithi112:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T00:57:07.195 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:57:07.195 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T00:57:07.377 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:57:07.377 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T00:57:07.589 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:57:07.589 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T00:57:07.757 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:57:07.757 DEBUG:teuthology.orchestra.run.smithi187:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T00:57:07.933 INFO:tasks.ceph:Archiving mon data... 2024-04-25T00:57:07.934 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/data/mon.a.tgz 2024-04-25T00:57:07.934 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2024-04-25T00:57:07.940 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.NEVwb4vV06 2024-04-25T00:57:07.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.NEVwb4vV06 2024-04-25T00:57:08.052 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.NEVwb4vV06 2024-04-25T00:57:08.116 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.NEVwb4vV06 is 264KB 2024-04-25T00:57:08.177 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.NEVwb4vV06 2024-04-25T00:57:08.182 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/data/mon.b.tgz 2024-04-25T00:57:08.182 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-25T00:57:08.188 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.7E93DrEjtz 2024-04-25T00:57:08.189 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7E93DrEjtz 2024-04-25T00:57:08.311 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.7E93DrEjtz 2024-04-25T00:57:08.374 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.7E93DrEjtz is 356KB 2024-04-25T00:57:08.439 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.7E93DrEjtz 2024-04-25T00:57:08.444 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/data/mon.c.tgz 2024-04-25T00:57:08.445 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2024-04-25T00:57:08.494 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.vs8tVU4C3T 2024-04-25T00:57:08.494 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vs8tVU4C3T 2024-04-25T00:57:08.618 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.vs8tVU4C3T 2024-04-25T00:57:08.683 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.vs8tVU4C3T is 356KB 2024-04-25T00:57:08.743 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.vs8tVU4C3T 2024-04-25T00:57:08.750 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T00:57:08.750 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:57:08.753 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T00:57:08.811 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:57:08.854 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T00:57:08.854 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T00:57:08.856 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/remote/smithi112/crash 2024-04-25T00:57:08.856 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:57:08.869 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/remote/smithi187/crash 2024-04-25T00:57:08.870 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T00:57:08.910 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T00:57:08.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:57:08.916 DEBUG:teuthology.orchestra.run.smithi187:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T00:57:08.989 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:57:09.002 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T00:57:09.045 INFO:tasks.ceph:Compressing logs... 2024-04-25T00:57:09.046 DEBUG:teuthology.orchestra.run.smithi112:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:57:09.048 DEBUG:teuthology.orchestra.run.smithi187:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:57:09.064 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27717.log 2024-04-25T00:57:09.065 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33843.log 2024-04-25T00:57:09.065 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.27717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33841.log 2024-04-25T00:57:09.065 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27717.log.gz 2024-04-25T00:57:09.066 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36174.log 2024-04-25T00:57:09.071 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33843.log.gz 2024-04-25T00:57:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36122.log 2024-04-25T00:57:09.071 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33841.log: /var/log/ceph/ceph-client.admin.36174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35380.log 2024-04-25T00:57:09.071 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33841.log.gz 2024-04-25T00:57:09.071 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36174.log.gz 2024-04-25T00:57:09.072 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36122.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T00:57:09.072 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36711.log 2024-04-25T00:57:09.072 INFO:teuthology.orchestra.run.smithi112.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.36122.log.gz 2024-04-25T00:57:09.072 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34494.log 2024-04-25T00:57:09.072 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35380.log.gz 2024-04-25T00:57:09.072 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36304.log 2024-04-25T00:57:09.072 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36711.log.gz 2024-04-25T00:57:09.072 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35731.log 2024-04-25T00:57:09.073 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34494.log.gz 2024-04-25T00:57:09.073 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36787.log 2024-04-25T00:57:09.073 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33504.log 2024-04-25T00:57:09.073 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36304.log.gz 2024-04-25T00:57:09.073 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36787.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35731.log.gz 2024-04-25T00:57:09.074 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34312.log 2024-04-25T00:57:09.074 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36787.log.gz 2024-04-25T00:57:09.074 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33504.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T00:57:09.074 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37121.log 2024-04-25T00:57:09.074 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33504.log.gz 2024-04-25T00:57:09.074 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.0.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34312.log.gz 2024-04-25T00:57:09.074 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35543.log 2024-04-25T00:57:09.074 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35198.log 2024-04-25T00:57:09.075 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36581.log 2024-04-25T00:57:09.075 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37121.log.gz 2024-04-25T00:57:09.075 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35543.log.gz 2024-04-25T00:57:09.085 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35915.log 2024-04-25T00:57:09.086 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35198.log: /var/log/ceph/ceph-client.admin.36581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36685.log 2024-04-25T00:57:09.086 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35198.log.gz 2024-04-25T00:57:09.086 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36581.log.gz 2024-04-25T00:57:09.086 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35622.log 2024-04-25T00:57:09.086 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.36685.log: /var/log/ceph/ceph-client.admin.33292.log 2024-04-25T00:57:09.087 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35915.log.gz 2024-04-25T00:57:09.087 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35622.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36685.log.gz 2024-04-25T00:57:09.087 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33740.log 2024-04-25T00:57:09.087 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33292.log: gzip 88.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.35622.log.gz -- 2024-04-25T00:57:09.087 INFO:teuthology.orchestra.run.smithi112.stderr: /var/log/ceph/ceph-client.admin.33792.log 2024-04-25T00:57:09.087 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33292.log.gz 2024-04-25T00:57:09.087 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36555.log 2024-04-25T00:57:09.087 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33766.log 2024-04-25T00:57:09.088 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8%/var/log/ceph/ceph-client.admin.33792.log: -- replaced with /var/log/ceph/ceph-client.admin.33740.log.gz 2024-04-25T00:57:09.088 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34964.log 2024-04-25T00:57:09.088 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33792.log.gz 2024-04-25T00:57:09.088 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36555.log.gz 2024-04-25T00:57:09.088 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33216.log 2024-04-25T00:57:09.089 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34964.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33766.log.gz 2024-04-25T00:57:09.089 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37226.log 2024-04-25T00:57:09.089 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33216.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34964.log.gz 2024-04-25T00:57:09.089 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37649.log 2024-04-25T00:57:09.089 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33216.log.gz 2024-04-25T00:57:09.090 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34468.log 2024-04-25T00:57:09.090 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37649.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37226.log.gz 2024-04-25T00:57:09.090 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33838.log 2024-04-25T00:57:09.090 INFO:teuthology.orchestra.run.smithi112.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37649.log.gz 2024-04-25T00:57:09.090 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33610.log 2024-04-25T00:57:09.090 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34468.log: /var/log/ceph/ceph-client.admin.33838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37597.log 2024-04-25T00:57:09.090 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33838.log.gz 2024-04-25T00:57:09.091 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34468.log.gz 2024-04-25T00:57:09.091 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37727.log 2024-04-25T00:57:09.091 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33610.log: /var/log/ceph/ceph-client.admin.37597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35172.log 2024-04-25T00:57:09.091 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37597.log.gz -- replaced with /var/log/ceph/ceph-client.admin.33610.log.gz 2024-04-25T00:57:09.091 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T00:57:09.091 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22789.log 2024-04-25T00:57:09.091 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37727.log: /var/log/ceph/ceph-client.admin.35172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34034.log 2024-04-25T00:57:09.092 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35172.log.gz 2024-04-25T00:57:09.092 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37727.log.gz 2024-04-25T00:57:09.092 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T00:57:09.092 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.tmp-client.admin.22789.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22789.log.gz 2024-04-25T00:57:09.092 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35016.log 2024-04-25T00:57:09.092 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34034.log.gz 2024-04-25T00:57:09.092 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36410.log 2024-04-25T00:57:09.093 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33532.log 2024-04-25T00:57:09.093 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35016.log.gz 2024-04-25T00:57:09.093 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35354.log 2024-04-25T00:57:09.093 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36410.log: /var/log/ceph/ceph-client.admin.33532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34679.log 2024-04-25T00:57:09.093 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36410.log.gz 2024-04-25T00:57:09.093 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36252.log 2024-04-25T00:57:09.093 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33532.log.gz 2024-04-25T00:57:09.094 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35354.log.gz 2024-04-25T00:57:09.094 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34679.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T00:57:09.094 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.34679.log.gz 2024-04-25T00:57:09.094 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37427.log 2024-04-25T00:57:09.095 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36252.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34938.log 2024-04-25T00:57:09.095 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36252.log.gz 2024-04-25T00:57:09.095 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35705.log 2024-04-25T00:57:09.095 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34938.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37427.log.gz 2024-04-25T00:57:09.095 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37147.log 2024-04-25T00:57:09.095 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34938.log.gz 2024-04-25T00:57:09.095 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36076.log 2024-04-25T00:57:09.096 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35705.log: /var/log/ceph/ceph-client.admin.37147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37330.log 2024-04-25T00:57:09.096 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35705.log.gz 2024-04-25T00:57:09.134 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27728.log 2024-04-25T00:57:09.134 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T00:57:09.134 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27752.log 2024-04-25T00:57:09.134 INFO:teuthology.orchestra.run.smithi187.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27728.log.gz 2024-04-25T00:57:09.134 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27848.log 2024-04-25T00:57:09.134 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27752.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27848.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27752.log.gz 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27776.log 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27848.log.gz 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27800.log 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27776.log.gz 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27800.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:57:09.135 INFO:teuthology.orchestra.run.smithi187.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27800.log.gz 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27824.log 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27824.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr: 87.8%/var/log/ceph/ceph-mon.c.log: 91.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27824.log.gz 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27704.log 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.27704.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T00:57:09.136 INFO:teuthology.orchestra.run.smithi187.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27704.log.gz/var/log/ceph/ceph-mds.b.log: 2024-04-25T00:57:09.137 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27680.log 2024-04-25T00:57:09.137 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-mon.b.log: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T00:57:09.137 INFO:teuthology.orchestra.run.smithi187.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T00:57:09.137 INFO:teuthology.orchestra.run.smithi187.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T00:57:09.137 INFO:teuthology.orchestra.run.smithi187.stderr:/var/log/ceph/ceph-client.admin.27680.log: 88.1%/var/log/ceph/ceph-osd.4.log: -- replaced with /var/log/ceph/ceph-client.admin.27680.log.gz 2024-04-25T00:57:09.145 INFO:teuthology.orchestra.run.smithi187.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T00:57:09.146 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37147.log.gz 2024-04-25T00:57:09.146 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 2024-04-25T00:57:09.146 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36076.log: /var/log/ceph/ceph-client.admin.37330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27836.log 2024-04-25T00:57:09.146 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36076.log.gz 2024-04-25T00:57:09.146 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37330.log.gz 2024-04-25T00:57:09.146 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38013.log 2024-04-25T00:57:09.147 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-04-25T00:57:09.147 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.27836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34807.log 2024-04-25T00:57:09.147 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.38013.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27836.log.gz 2024-04-25T00:57:09.147 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37479.log 2024-04-25T00:57:09.147 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38013.log.gz 2024-04-25T00:57:09.147 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.34807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27601.log 2024-04-25T00:57:09.147 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35120.log 2024-04-25T00:57:09.147 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.27601.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37479.log.gz 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.0.34807.log.gz 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36607.log 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33148.log 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35120.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.27601.log.gz 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36607.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35120.log.gz 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34047.log 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36607.log.gz 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35785.log 2024-04-25T00:57:09.148 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33148.log: /var/log/ceph/ceph-client.admin.34047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37095.log 2024-04-25T00:57:09.149 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33148.log.gz 2024-04-25T00:57:09.149 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34047.log.gz 2024-04-25T00:57:09.149 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37569.log 2024-04-25T00:57:09.149 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35785.log.gz 2024-04-25T00:57:09.149 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35596.log 2024-04-25T00:57:09.149 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37095.log.gz 2024-04-25T00:57:09.149 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35759.log 2024-04-25T00:57:09.149 INFO:teuthology.orchestra.run.smithi112.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37569.log.gz 2024-04-25T00:57:09.150 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37987.log 2024-04-25T00:57:09.150 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-04-25T00:57:09.150 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35596.log.gz 2024-04-25T00:57:09.150 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35759.log.gz 2024-04-25T00:57:09.150 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37883.log 2024-04-25T00:57:09.150 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36438.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34049.log 2024-04-25T00:57:09.150 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37987.log.gz 2024-04-25T00:57:09.150 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33714.log 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37883.log.gz 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36960.log 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34049.log: /var/log/ceph/ceph-client.admin.33714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37038.log 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34049.log.gz 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33714.log.gz 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34912.log 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36960.log.gz 2024-04-25T00:57:09.151 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37252.log 2024-04-25T00:57:09.152 INFO:teuthology.orchestra.run.smithi112.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37038.log.gz 2024-04-25T00:57:09.152 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37364.log 2024-04-25T00:57:09.152 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36278.log 2024-04-25T00:57:09.152 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34912.log.gz 2024-04-25T00:57:09.152 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37364.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37252.log.gz 2024-04-25T00:57:09.152 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36659.log 2024-04-25T00:57:09.152 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36278.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37364.log.gz 2024-04-25T00:57:09.152 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34364.log 2024-04-25T00:57:09.153 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37517.log 2024-04-25T00:57:09.153 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36278.log.gz 2024-04-25T00:57:09.153 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34364.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36659.log.gz 2024-04-25T00:57:09.153 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35648.log 2024-04-25T00:57:09.153 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37517.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T00:57:09.153 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34364.log.gz 2024-04-25T00:57:09.154 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3%/var/log/ceph/ceph-client.admin.35648.log: -- replaced with /var/log/ceph/ceph-client.admin.37517.log.gz 2024-04-25T00:57:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36356.log 2024-04-25T00:57:09.154 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35648.log.gz 2024-04-25T00:57:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34572.log 2024-04-25T00:57:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35406.log 2024-04-25T00:57:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34572.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36356.log.gz 2024-04-25T00:57:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33688.log 2024-04-25T00:57:09.154 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36843.log 2024-04-25T00:57:09.155 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34572.log.gz 2024-04-25T00:57:09.155 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35406.log: /var/log/ceph/ceph-client.admin.33688.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35406.log.gz 2024-04-25T00:57:09.155 INFO:teuthology.orchestra.run.smithi112.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33688.log.gz 2024-04-25T00:57:09.155 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37857.log 2024-04-25T00:57:09.155 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.0.36843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35570.log 2024-04-25T00:57:09.155 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37857.log: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35889.log 2024-04-25T00:57:09.156 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T00:57:09.156 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37857.log.gz 2024-04-25T00:57:09.156 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35146.log 2024-04-25T00:57:09.156 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38089.log 2024-04-25T00:57:09.156 INFO:teuthology.orchestra.run.smithi112.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35570.log.gz/var/log/ceph/ceph-client.admin.35146.log: 2024-04-25T00:57:09.156 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35889.log.gz 2024-04-25T00:57:09.156 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35146.log.gz 2024-04-25T00:57:09.156 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37701.log 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33448.log 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.38089.log: /var/log/ceph/ceph-client.admin.37701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36503.log 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38089.log.gz 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.36843.log.gz 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37701.log.gz 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33448.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33448.log.gz 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37543.log 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37753.log 2024-04-25T00:57:09.157 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36503.log.gz 2024-04-25T00:57:09.158 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34416.log 2024-04-25T00:57:09.158 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33558.log 2024-04-25T00:57:09.158 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37543.log.gz 2024-04-25T00:57:09.158 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37753.log: /var/log/ceph/ceph-client.admin.34416.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.37753.log.gz -5 2024-04-25T00:57:09.158 INFO:teuthology.orchestra.run.smithi112.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37278.log 2024-04-25T00:57:09.158 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.34416.log.gz 2024-04-25T00:57:09.158 INFO:teuthology.orchestra.run.smithi112.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T00:57:09.158 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33558.log: /var/log/ceph/ceph-client.admin.37278.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33558.log.gz 2024-04-25T00:57:09.159 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35276.log 2024-04-25T00:57:09.159 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33396.log 2024-04-25T00:57:09.159 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37278.log.gz 2024-04-25T00:57:09.159 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.35276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38065.log 2024-04-25T00:57:09.159 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33636.log 2024-04-25T00:57:09.159 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35276.log.gz 2024-04-25T00:57:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33396.log: /var/log/ceph/ceph-client.admin.38065.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33396.log.gz 2024-04-25T00:57:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37069.log 2024-04-25T00:57:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35863.log 2024-04-25T00:57:09.160 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38065.log.gz 2024-04-25T00:57:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37069.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33636.log.gz 2024-04-25T00:57:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35068.log 2024-04-25T00:57:09.160 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37069.log.gz 2024-04-25T00:57:09.160 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35302.log 2024-04-25T00:57:09.161 INFO:teuthology.orchestra.run.smithi112.stderr: 88.6%/var/log/ceph/ceph-client.admin.35068.log: -- replaced with /var/log/ceph/ceph-client.admin.35863.log.gz 2024-04-25T00:57:09.161 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33318.log 2024-04-25T00:57:09.161 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35068.log.gz 2024-04-25T00:57:09.161 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35302.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35302.log.gz 2024-04-25T00:57:09.161 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34053.log 2024-04-25T00:57:09.161 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35674.log 2024-04-25T00:57:09.161 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4%/var/log/ceph/ceph-client.admin.34053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34886.log 2024-04-25T00:57:09.161 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33318.log.gz 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34053.log.gz 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35674.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35674.log.gz 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34624.log 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35439.log 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34624.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34886.log.gz 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33476.log 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35439.log: /var/log/ceph/ceph-client.admin.33476.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.34624.log.gz 2024-04-25T00:57:09.162 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35439.log.gz 2024-04-25T00:57:09.163 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33476.log.gz 2024-04-25T00:57:09.163 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36382.log 2024-04-25T00:57:09.163 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37805.log 2024-04-25T00:57:09.163 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36473.log 2024-04-25T00:57:09.163 INFO:teuthology.orchestra.run.smithi112.stderr: 83.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T00:57:09.163 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37805.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36382.log.gz 2024-04-25T00:57:09.163 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37805.log.gz 2024-04-25T00:57:09.163 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35465.log 2024-04-25T00:57:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33246.log 2024-04-25T00:57:09.164 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8%/var/log/ceph/ceph-client.admin.35465.log: -- replaced with /var/log/ceph/ceph-client.admin.36473.log.gz 2024-04-25T00:57:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36633.log 2024-04-25T00:57:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34860.log 2024-04-25T00:57:09.164 INFO:teuthology.orchestra.run.smithi112.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35465.log.gz 2024-04-25T00:57:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33246.log: /var/log/ceph/ceph-client.admin.36633.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33246.log.gz 2024-04-25T00:57:09.164 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36633.log.gz 2024-04-25T00:57:09.164 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34234.log 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9%/var/log/ceph/ceph-client.admin.34234.log: -- replaced with /var/log/ceph/ceph-client.admin.34860.log.gz 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36021.log 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37935.log 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34234.log.gz 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35042.log: /var/log/ceph/ceph-client.admin.36021.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36021.log.gz 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34598.log 2024-04-25T00:57:09.165 INFO:teuthology.orchestra.run.smithi112.stderr:gzip/var/log/ceph/ceph-client.admin.37935.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33833.log 2024-04-25T00:57:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34598.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37675.log 2024-04-25T00:57:09.166 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37935.log.gz 2024-04-25T00:57:09.166 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34598.log.gz 2024-04-25T00:57:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33833.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33833.log.gz 2024-04-25T00:57:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33344.log 2024-04-25T00:57:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33836.log 2024-04-25T00:57:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33344.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37675.log.gz 2024-04-25T00:57:09.166 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36986.log 2024-04-25T00:57:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34990.log 2024-04-25T00:57:09.167 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33344.log.gz 2024-04-25T00:57:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33836.log: /var/log/ceph/ceph-client.admin.36986.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36986.log.gz 2024-04-25T00:57:09.167 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33836.log.gz 2024-04-25T00:57:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35094.log 2024-04-25T00:57:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37304.log 2024-04-25T00:57:09.167 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3%/var/log/ceph/ceph-client.admin.35094.log: -- replaced with /var/log/ceph/ceph-client.admin.34990.log.gz 2024-04-25T00:57:09.167 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36226.log 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35094.log.gz 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37304.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37304.log.gz 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34520.log 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35967.log 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34520.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36226.log.gz 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35811.log 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35250.log 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34520.log.gz 2024-04-25T00:57:09.168 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35967.log: /var/log/ceph/ceph-client.admin.35811.log: 88.9% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35811.log.gz -- replaced with /var/log/ceph/ceph-client.admin.35967.log.gz 2024-04-25T00:57:09.169 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T00:57:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T00:57:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34546.log 2024-04-25T00:57:09.169 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.35250.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35491.log 2024-04-25T00:57:09.169 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T00:57:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34546.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34546.log.gz 2024-04-25T00:57:09.169 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T00:57:09.169 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz/var/log/ceph/ceph-client.admin.35491.log: 2024-04-25T00:57:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35941.log 2024-04-25T00:57:09.170 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34390.log 2024-04-25T00:57:09.170 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35491.log.gz 2024-04-25T00:57:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.35941.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35941.log.gz 2024-04-25T00:57:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-04-25T00:57:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37012.log 2024-04-25T00:57:09.170 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37173.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34390.log.gz 2024-04-25T00:57:09.170 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-04-25T00:57:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35328.log 2024-04-25T00:57:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36737.log 2024-04-25T00:57:09.171 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5%/var/log/ceph/ceph-client.admin.35328.log: -- replaced with /var/log/ceph/ceph-client.admin.37012.log.gz 2024-04-25T00:57:09.171 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34051.log 2024-04-25T00:57:09.171 INFO:teuthology.orchestra.run.smithi112.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35328.log.gz 2024-04-25T00:57:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35995.log 2024-04-25T00:57:09.171 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34051.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.36737.log.gz 2024-04-25T00:57:09.171 INFO:teuthology.orchestra.run.smithi112.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34051.log.gz 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38039.log 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34054.log 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr: 89.3%/var/log/ceph/ceph-client.admin.38039.log: -- replaced with /var/log/ceph/ceph-client.admin.35995.log.gz 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38039.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37200.log 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37909.log 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34054.log.gz 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37623.log 2024-04-25T00:57:09.172 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37909.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37200.log.gz 2024-04-25T00:57:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37831.log 2024-04-25T00:57:09.173 INFO:teuthology.orchestra.run.smithi112.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37909.log.gz 2024-04-25T00:57:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35224.log 2024-04-25T00:57:09.173 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37831.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37623.log.gz 2024-04-25T00:57:09.185 INFO:teuthology.orchestra.run.smithi112.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37831.log.gz 2024-04-25T00:57:09.185 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36330.log 2024-04-25T00:57:09.186 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33830.log 2024-04-25T00:57:09.186 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35224.log: /var/log/ceph/ceph-client.admin.36330.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35224.log.gz 2024-04-25T00:57:09.186 INFO:teuthology.orchestra.run.smithi112.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36330.log.gz 2024-04-25T00:57:09.186 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34052.log 2024-04-25T00:57:09.186 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36529.log 2024-04-25T00:57:09.186 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33830.log: /var/log/ceph/ceph-client.admin.34052.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33830.log.gz 2024-04-25T00:57:09.186 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34052.log.gz 2024-04-25T00:57:09.186 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38115.log 2024-04-25T00:57:09.187 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34045.log 2024-04-25T00:57:09.187 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36529.log: /var/log/ceph/ceph-client.admin.38115.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36529.log.gz 2024-04-25T00:57:09.187 INFO:teuthology.orchestra.run.smithi112.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38115.log.gz 2024-04-25T00:57:09.187 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36904.log 2024-04-25T00:57:09.187 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34338.log 2024-04-25T00:57:09.187 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34045.log: 88.5%/var/log/ceph/ceph-client.admin.36904.log: -- replaced with /var/log/ceph/ceph-client.admin.34045.log.gz 2024-04-25T00:57:09.187 INFO:teuthology.orchestra.run.smithi112.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36904.log.gz 2024-04-25T00:57:09.187 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36934.log 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33845.log 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr: 89.1%/var/log/ceph/ceph-client.admin.36934.log: -- replaced with /var/log/ceph/ceph-client.admin.34338.log.gz 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36934.log.gz 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33370.log 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33845.log.gz 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33370.log.gz 2024-04-25T00:57:09.188 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37961.log 2024-04-25T00:57:09.189 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-04-25T00:57:09.189 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35517.log 2024-04-25T00:57:09.189 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37961.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz 2024-04-25T00:57:09.189 INFO:teuthology.orchestra.run.smithi112.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37961.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33846.log 2024-04-25T00:57:09.189 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T00:57:09.189 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33584.log 2024-04-25T00:57:09.189 INFO:teuthology.orchestra.run.smithi112.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T00:57:09.189 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.35517.log: /var/log/ceph/ceph-client.admin.33846.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35517.log.gz 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33846.log.gz 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33422.log 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36200.log 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.33584.log: /var/log/ceph/ceph-client.admin.33422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34286.log 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33584.log.gz 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33422.log.gz 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36200.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36200.log.gz 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33662.log 2024-04-25T00:57:09.190 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37779.log 2024-04-25T00:57:09.191 INFO:teuthology.orchestra.run.smithi112.stderr: 89.0%/var/log/ceph/ceph-client.admin.33662.log: -- replaced with /var/log/ceph/ceph-client.admin.34286.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37453.log 2024-04-25T00:57:09.191 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T00:57:09.191 INFO:teuthology.orchestra.run.smithi112.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33662.log.gz 2024-04-25T00:57:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37779.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37779.log.gz 2024-04-25T00:57:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34260.log 2024-04-25T00:57:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.37453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36148.log 2024-04-25T00:57:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34260.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37453.log.gz 2024-04-25T00:57:09.191 INFO:teuthology.orchestra.run.smithi112.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34442.log 2024-04-25T00:57:09.192 INFO:teuthology.orchestra.run.smithi112.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34260.log.gz 2024-04-25T00:57:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.36148.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36148.log.gz 2024-04-25T00:57:09.192 INFO:teuthology.orchestra.run.smithi112.stderr:/var/log/ceph/ceph-client.admin.34442.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34442.log.gz 2024-04-25T00:57:09.192 INFO:teuthology.orchestra.run.smithi112.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T00:57:09.192 INFO:teuthology.orchestra.run.smithi112.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T00:57:09.192 INFO:teuthology.orchestra.run.smithi187.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T00:57:09.193 INFO:teuthology.orchestra.run.smithi187.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T00:57:09.223 INFO:teuthology.orchestra.run.smithi112.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T00:57:09.224 INFO:teuthology.orchestra.run.smithi112.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T00:57:09.234 INFO:teuthology.orchestra.run.smithi187.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T00:57:09.235 INFO:teuthology.orchestra.run.smithi187.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T00:57:09.241 INFO:teuthology.orchestra.run.smithi187.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T00:57:09.246 INFO:teuthology.orchestra.run.smithi187.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T00:57:09.248 INFO:teuthology.orchestra.run.smithi187.stderr: 2024-04-25T00:57:09.248 INFO:teuthology.orchestra.run.smithi187.stderr:real 0m0.148s 2024-04-25T00:57:09.248 INFO:teuthology.orchestra.run.smithi187.stderr:user 0m0.597s 2024-04-25T00:57:09.248 INFO:teuthology.orchestra.run.smithi187.stderr:sys 0m0.047s 2024-04-25T00:57:09.264 INFO:teuthology.orchestra.run.smithi112.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T00:57:09.265 INFO:teuthology.orchestra.run.smithi112.stderr: 2024-04-25T00:57:09.265 INFO:teuthology.orchestra.run.smithi112.stderr:real 0m0.215s 2024-04-25T00:57:09.265 INFO:teuthology.orchestra.run.smithi112.stderr:user 0m0.806s 2024-04-25T00:57:09.266 INFO:teuthology.orchestra.run.smithi112.stderr:sys 0m0.136s 2024-04-25T00:57:09.266 INFO:tasks.ceph:Archiving logs... 2024-04-25T00:57:09.266 DEBUG:teuthology.misc:Transferring archived files from smithi112:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/remote/smithi112/log 2024-04-25T00:57:09.267 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:57:09.523 DEBUG:teuthology.misc:Transferring archived files from smithi187:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/remote/smithi187/log 2024-04-25T00:57:09.524 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:57:09.617 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:57:09.627 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:57:09.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:57:09.630 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:57:09.712 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:57:09.722 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:57:09.722 DEBUG:teuthology.orchestra.run.smithi112:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:57:09.725 DEBUG:teuthology.orchestra.run.smithi187:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:57:09.735 INFO:teuthology.orchestra.run.smithi112.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:57:09.735 INFO:teuthology.orchestra.run.smithi112.stdout:============================================================================== 2024-04-25T00:57:09.735 INFO:teuthology.orchestra.run.smithi112.stdout:*hv01.front.sepi 44.4.53.2 2 u 56 64 37 0.079 -2.453 1.016 2024-04-25T00:57:09.735 INFO:teuthology.orchestra.run.smithi112.stdout:+hv02.front.sepi 63.231.80.2 3 u 56 64 37 0.078 -1.341 1.026 2024-04-25T00:57:09.735 INFO:teuthology.orchestra.run.smithi112.stdout:+hv03.front.sepi 74.6.168.72 3 u 58 64 37 0.127 -1.605 1.096 2024-04-25T00:57:09.735 INFO:teuthology.orchestra.run.smithi112.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:57:09.738 INFO:teuthology.orchestra.run.smithi187.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T00:57:09.738 INFO:teuthology.orchestra.run.smithi187.stdout:============================================================================== 2024-04-25T00:57:09.738 INFO:teuthology.orchestra.run.smithi187.stdout:*hv01.front.sepi 44.4.53.2 2 u 54 64 37 0.130 -1.493 0.495 2024-04-25T00:57:09.738 INFO:teuthology.orchestra.run.smithi187.stdout:+hv02.front.sepi 63.231.80.2 3 u 57 64 37 0.103 -0.400 0.514 2024-04-25T00:57:09.738 INFO:teuthology.orchestra.run.smithi187.stdout:+hv03.front.sepi 74.6.168.72 3 u 58 64 37 0.120 -0.662 0.376 2024-04-25T00:57:09.739 INFO:teuthology.orchestra.run.smithi187.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T00:57:09.739 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:57:09.748 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:57:09.749 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:57:09.758 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:57:09.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:57:09.776 INFO:teuthology.task.internal:Duration was 746.806605 seconds 2024-04-25T00:57:09.776 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:57:09.785 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:57:09.786 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:57:09.788 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:57:09.825 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:57:09.825 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi112.front.sepia.ceph.com 2024-04-25T00:57:09.825 DEBUG:teuthology.orchestra.run.smithi112:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:57:09.876 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi187.front.sepia.ceph.com 2024-04-25T00:57:09.876 DEBUG:teuthology.orchestra.run.smithi187:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:57:09.886 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:57:09.887 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:57:09.920 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:57:09.954 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:57:09.955 DEBUG:teuthology.orchestra.run.smithi112:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:57:09.988 DEBUG:teuthology.orchestra.run.smithi187:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:57:10.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:57:10.062 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:57:10.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:57:10.073 DEBUG:teuthology.orchestra.run.smithi187:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:57:10.086 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:57:10.096 DEBUG:teuthology.orchestra.run.smithi112:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:57:10.116 DEBUG:teuthology.orchestra.run.smithi187:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:57:10.123 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = core 2024-04-25T00:57:10.137 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = core 2024-04-25T00:57:10.169 DEBUG:teuthology.orchestra.run.smithi112:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:57:10.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:57:10.182 DEBUG:teuthology.orchestra.run.smithi187:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:57:10.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:57:10.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:57:10.228 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:57:10.228 DEBUG:teuthology.misc:Transferring archived files from smithi112:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/remote/smithi112 2024-04-25T00:57:10.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:57:10.248 DEBUG:teuthology.misc:Transferring archived files from smithi187:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285/remote/smithi187 2024-04-25T00:57:10.249 DEBUG:teuthology.orchestra.run.smithi187:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:57:10.279 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:57:10.280 DEBUG:teuthology.orchestra.run.smithi112:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:57:10.288 DEBUG:teuthology.orchestra.run.smithi187:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:57:10.326 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:57:10.337 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:57:10.337 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:57:10.349 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:57:10.350 DEBUG:teuthology.orchestra.run.smithi112:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:57:10.352 DEBUG:teuthology.orchestra.run.smithi187:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:57:10.355 INFO:teuthology.orchestra.run.smithi112.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:57 /home/ubuntu/cephtest 2024-04-25T00:57:10.373 INFO:teuthology.orchestra.run.smithi187.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:57 /home/ubuntu/cephtest 2024-04-25T00:57:10.375 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:57:10.385 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:57:10.405 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T00:57:10.431 DEBUG:teuthology.nuke:shortname: smithi112 2024-04-25T00:57:10.432 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:57:10.455 DEBUG:teuthology.nuke:shortname: smithi187 2024-04-25T00:57:10.455 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:57:10.459 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:36:06.856315', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ4w3XTBD3z3iI/FJ0TDJz24aQSOg0wgpZEJWbM2zwmoU5cppJBwtxtTf2VrZri4GLiwfDKYSxwe32Eb7UzdZk8='} 2024-04-25T00:57:10.483 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672285', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:36:06.858043', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNMI9STWpOkq+7tF/jBvaEKwDNQ9dzEZTLukGTCCNQDFKQkri1l5VVzrOTwYMlImxhfXnlgrD3G/FfhHvX3p0Kg='} 2024-04-25T00:57:10.513 INFO:teuthology.orchestra.console.smithi112:Power off 2024-04-25T00:57:10.513 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T00:57:10.537 DEBUG:teuthology.orchestra.console.smithi112:power off output: Chassis Power Control: Down/Off 2024-04-25T00:57:10.537 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:57:10.555 INFO:teuthology.orchestra.console.smithi187:Power off 2024-04-25T00:57:10.555 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T00:57:10.566 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-25T00:57:10.578 DEBUG:teuthology.orchestra.console.smithi187:power off output: Chassis Power Control: Down/Off 2024-04-25T00:57:10.578 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:57:10.600 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-25T00:57:14.568 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:57:14.601 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:57:14.679 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is on 2024-04-25T00:57:14.712 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is on 2024-04-25T00:57:18.683 DEBUG:teuthology.orchestra.console.smithi112:pexpect command: ipmitool -H smithi112.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:57:18.713 DEBUG:teuthology.orchestra.console.smithi187:pexpect command: ipmitool -H smithi187.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T00:57:18.794 DEBUG:teuthology.orchestra.console.smithi112:check power output: Chassis Power is off 2024-04-25T00:57:18.826 DEBUG:teuthology.orchestra.console.smithi187:check power output: Chassis Power is off 2024-04-25T00:57:18.895 INFO:teuthology.orchestra.console.smithi112:Power off completed 2024-04-25T00:57:18.927 INFO:teuthology.orchestra.console.smithi187:Power off completed 2024-04-25T00:57:19.032 INFO:teuthology.run:Summary data: description: fs/shell/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1-mds-1-client-coloc conf/{client mds mgr mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/cephfs-shell} duration: 746.8066048622131 failure_reason: 'Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=127d8d3c36ea41b380b54405929e021b status: fail success: false 2024-04-25T00:57:19.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:57:19.130 INFO:teuthology.run:FAIL